Jul 01 19:21:27.615275 ip-10-0-138-218 systemd[1]: Starting Kubernetes Kubelet...
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.012707    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016890    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016910    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016914    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016917    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016924    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016928    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016930    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 19:21:28.190552 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016936    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 19:21:28.086341 ip-10-0-138-218 systemd[1]: Started Kubernetes Kubelet.
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016940    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016942    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016945    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016966    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016969    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016972    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016975    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016978    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016980    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016987    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.016990    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017005    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017012    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017015    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017020    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017023    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017026    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017029    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017036    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 19:21:28.200502 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017039    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017041    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017046    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017048    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017051    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017053    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017060    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017063    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017065    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017070    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017073    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017076    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017078    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017085    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017087    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017090    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017095    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017098    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017100    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017103    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 19:21:28.201402 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017110    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017113    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017120    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017124    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017127    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017134    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017137    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017139    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017144    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017147    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017150    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017152    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017158    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017161    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017164    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017166    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017171    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017174    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017177    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017183    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 19:21:28.202108 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017186    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017188    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017190    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017193    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017197    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017199    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017202    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017208    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017211    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017213    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017219    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017222    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017227    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017235    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017238    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017244    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017247    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017250    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017257    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 19:21:28.202727 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017260    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017263    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017268    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017271    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017273    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017276    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017282    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017286    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017290    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017296    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017299    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017301    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.017308    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018213    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018220    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018225    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018227    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018230    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018233    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 19:21:28.203303 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018239    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018242    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018246    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018252    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018255    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018258    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018260    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018263    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018266    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018272    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018274    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018277    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018282    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018284    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018287    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018290    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018295    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018298    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018300    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 19:21:28.204319 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018305    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018310    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018316    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018320    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018322    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018328    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018331    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018334    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018338    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018341    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018343    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018351    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018355    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018361    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018364    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018368    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018373    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018376    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018378    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 19:21:28.204907 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018383    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018386    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018389    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018395    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018399    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018402    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018407    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018410    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018412    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018417    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018420    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018422    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018425    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018430    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018433    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018435    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018440    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018443    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018445    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018450    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 19:21:28.205682 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018453    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018456    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018458    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018461    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018464    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018466    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018473    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018476    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018481    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018483    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018486    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018488    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018492    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018495    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018498    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018500    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018504    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018507    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018510    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018515    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 19:21:28.206400 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018518    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018520    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018523    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018528    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018530    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018533    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018538    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018540    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018543    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018546    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018551    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018557    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018562    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018564    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.018567    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019133    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019141    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019145    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019150    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019154    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 19:21:28.207081 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019157    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019162    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019165    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019167    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019172    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019175    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019177    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019182    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019185    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019187    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019190    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019195    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019198    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019200    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019204    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019207    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019210    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019215    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019219    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 19:21:28.207801 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019222    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019227    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019230    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019235    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019239    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019242    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019245    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019248    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019251    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019253    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019258    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019261    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019263    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019269    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019271    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019274    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019278    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019281    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019284    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019286    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 19:21:28.208418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019291    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019294    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019300    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019303    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019305    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019308    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019313    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019315    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019318    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019322    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019325    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019328    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019332    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019335    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019337    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019339    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019347    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019349    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019354    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019357    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 19:21:28.209052 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019359    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019362    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019366    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019369    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019372    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019376    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019379    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019382    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019386    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019389    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019391    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019394    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019400    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019403    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019408    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019410    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019413    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019416    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019421    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019423    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 19:21:28.209599 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019426    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019430    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019433    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019435    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019438    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019441    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019444    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019446    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019452    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019455    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019457    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019462    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019464    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019467    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.019472    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019551    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019568    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019576    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019583    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019587    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019593    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 01 19:21:28.210177 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019597    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019606    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019609    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019612    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019617    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019621    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019625    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019628    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019632    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019634    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019638    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019641    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019647    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019650    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019653    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019658    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019662    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019665    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019670    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019674    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019679    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019682    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019685    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019690    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 01 19:21:28.210722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019694    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019697    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019703    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019706    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019709    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019714    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019717    2581 flags.go:64] FLAG: --enable-server="true"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019729    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019733    2581 flags.go:64] FLAG: --event-burst="100"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019740    2581 flags.go:64] FLAG: --event-qps="50"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019743    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019747    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019750    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019759    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019762    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019765    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019770    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019773    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019787    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019793    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019799    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019802    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019805    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019810    2581 flags.go:64] FLAG: --feature-gates=""
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019814    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 01 19:21:28.211385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019819    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019822    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019828    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019831    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019834    2581 flags.go:64] FLAG: --help="false"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019837    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019840    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019843    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019846    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019851    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019855    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019858    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019862    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019865    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019868    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019874    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019879    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019884    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019887    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019889    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019894    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019897    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019900    2581 flags.go:64] FLAG: --lock-file=""
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019905    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 01 19:21:28.212316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019908    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019911    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019918    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019921    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019927    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019929    2581 flags.go:64] FLAG: --logging-format="text"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019932    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019937    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019940    2581 flags.go:64] FLAG: --manifest-url=""
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019943    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019985    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019991    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.019996    2581 flags.go:64] FLAG: --max-pods="110"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020001    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020004    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020008    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020011    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020014    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020017    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020020    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020029    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020032    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020035    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 01 19:21:28.213029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020039    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020044    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020052    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020055    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020060    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020063    2581 flags.go:64] FLAG: --port="10250"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020066    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020069    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f76da472f2361bdc"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020074    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020077    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020082    2581 flags.go:64] FLAG: --register-node="true"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020085    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020089    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020094    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020097    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020099    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020105    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020108    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020111    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020116    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020120    2581 flags.go:64] FLAG: --runonce="false"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020123    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020128    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020131    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020134    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 01 19:21:28.213641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020139    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020142    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020146    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020150    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020152    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020155    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020158    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020161    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020165    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020169    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020172    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020178    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020181    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020184    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020188    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020191    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020194    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020197    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020200    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020203    2581 flags.go:64] FLAG: --v="2"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020207    2581 flags.go:64] FLAG: --version="false"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020211    2581 flags.go:64] FLAG: --vmodule=""
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020215    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.020218    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020333    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 19:21:28.214358 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020337    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020341    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020347    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020350    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020353    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020356    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020359    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020362    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020364    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020367    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020369    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020372    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020374    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020377    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020380    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020384    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020386    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020389    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020392    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020395    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 19:21:28.452668 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020398    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020400    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020403    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020405    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020408    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020410    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020413    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020415    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020418    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020421    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020423    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020426    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020428    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020431    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020435    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020438    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020440    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020443    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020446    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020448    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 19:21:28.453758 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020451    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020454    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020456    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020459    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020462    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020464    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020467    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020471    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020473    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020476    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020479    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020482    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020484    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020487    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020490    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020492    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020495    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020498    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020500    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 19:21:28.454813 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020503    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020507    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020510    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020513    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020516    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020520    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020524    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020530    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020533    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020535    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020538    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020540    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020543    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020546    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020549    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020551    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020554    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020556    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020559    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 19:21:28.455418 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020561    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020565    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020568    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020571    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020573    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020576    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020579    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020581    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020584    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020587    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020589    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020592    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020594    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020597    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020600    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020602    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020605    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020608    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020610    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 19:21:28.456593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.020613    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.021323    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.027697    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.027715    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.027735    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.027741    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027807    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027813    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027816    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027819    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027822    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027824    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027827    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027830    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027833    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 19:21:28.457265 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027836    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027838    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027841    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027844    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027846    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027849    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027852    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027855    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027858    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027861    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027863    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027865    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027868    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027870    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027873    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027876    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027879    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027881    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027884    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027887    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 19:21:28.457759 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027891    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027893    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027898    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027901    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027904    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027907    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027910    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027914    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027917    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027920    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027923    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027926    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027929    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027931    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027934    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027938    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027941    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027945    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 19:21:28.458334 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027947    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027962    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027966    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027969    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027971    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027975    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027977    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027980    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027982    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027985    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027988    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027990    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027993    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027996    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.027998    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028001    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028004    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028007    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028009    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028012    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 19:21:28.458880 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028014    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028017    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028020    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028022    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028025    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028027    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028030    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028032    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028035    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028038    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028040    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028043    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028045    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028048    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028051    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028054    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028056    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028059    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028062    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028064    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 19:21:28.459454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028067    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028069    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028072    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028074    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028079    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028082    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028084    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028087    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028090    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028093    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028096    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028099    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.028104    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028237    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028241    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 19:21:28.460130 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028245    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028247    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028250    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028253    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028255    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028258    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028260    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028263    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028267    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028270    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028273    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028276    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028280    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028283    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028286    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028288    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028291    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028294    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028296    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 19:21:28.460578 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028299    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028301    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028305    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028307    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028310    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028312    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028315    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028318    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028320    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028323    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028325    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028328    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028330    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028333    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028335    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028338    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028340    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028343    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028346    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028348    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 19:21:28.461084 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028352    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028355    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028357    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028360    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028362    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028366    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028369    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028371    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028374    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028377    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028379    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028383    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028386    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028389    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028393    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028396    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028400    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028403    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028406    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 19:21:28.461645 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028408    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028411    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028414    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028416    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028419    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028422    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028424    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028426    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028429    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028431    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028434    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028436    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028439    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028441    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028444    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028446    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028464    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028468    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028474    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028477    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 19:21:28.462236 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028480    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028483    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028486    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028488    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028491    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028493    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028496    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028500    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028502    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028505    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028508    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028510    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028513    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028515    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028518    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028521    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028523    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028526    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 19:21:28.462778 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:28.028528    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 19:21:28.463424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.028533    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 19:21:28.463424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.028647    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 01 19:21:28.463424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.030697    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 01 19:21:28.463424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.031982    2581 server.go:1019] "Starting client certificate rotation"
Jul 01 19:21:28.463424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.032090    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 19:21:28.463424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.032136    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 19:21:28.463424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.035809    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 19:21:28.463424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.039751    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 19:21:28.463424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.050262    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 01 19:21:28.463424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.052421    2581 log.go:25] "Validated CRI v1 image API"
Jul 01 19:21:28.463424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.053263    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 01 19:21:28.463424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.058083    2581 fs.go:135] Filesystem UUIDs: map[5407bb8d-91b4-41e9-ac6b-d363f7f9f21a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ab3313f3-4954-43aa-aec7-1949897b20ee:/dev/nvme0n1p3]
Jul 01 19:21:28.463802 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.058099    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 01 19:21:28.463963 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.062326    2581 manager.go:216] Machine: {Timestamp:2026-07-01 19:21:28.061823037 +0000 UTC m=+0.331571155 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103041 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a15999f618c61f8d0dd6e0d3c5de4 SystemUUID:ec2a1599-9f61-8c61-f8d0-dd6e0d3c5de4 BootID:6118b901-8a81-4ec4-979b-fe557ec9ca8d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b4:2f:cf:9b:4b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b4:2f:cf:9b:4b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:85:80:0c:bf:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 01 19:21:28.463963 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.063463    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 01 19:21:28.463963 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.063593    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 01 19:21:28.463963 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.066580    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 01 19:21:28.463963 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.066646    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 19:21:28.463963 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.066608    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-218.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 01 19:21:28.463963 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.067429    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.067439    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.067449    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.068680    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.069720    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.072372    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.072489    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.072516    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.072533    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.072546    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.072556    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.072569    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.072935    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.072967    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.075147    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.076477    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.076504    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.077678    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.077693    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.077699    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.077705    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.077714    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.077720    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.077727    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.077732    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.077739    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.077746    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.077761    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.077767    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.078606    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.078739    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.084869    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-218.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.084895    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-218.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.085059    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.085460    2581 server.go:1268] "Started kubelet"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.085533    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.085803    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.085871    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.089048    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 01 19:21:28.464257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.089151    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.091343    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.092312    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-218.ec2.internal.18be40be3068a055  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-218.ec2.internal,UID:ip-10-0-138-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-218.ec2.internal,},FirstTimestamp:2026-07-01 19:21:28.085422165 +0000 UTC m=+0.355170282,LastTimestamp:2026-07-01 19:21:28.085422165 +0000 UTC m=+0.355170282,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-218.ec2.internal,}"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.096090    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.096164    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.096177    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-218.ec2.internal\" not found"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.096653    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.096692    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.096730    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.096864    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.096872    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.097223    2581 factory.go:55] Registering systemd factory
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.097238    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.097412    2581 factory.go:153] Registering CRI-O factory
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.097420    2581 factory.go:223] Registration of the crio container factory successfully
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.097471    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.097489    2581 factory.go:103] Registering Raw factory
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.097500    2581 manager.go:1196] Started watching for new ooms in manager
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.097769    2581 manager.go:349] Starting recovery of all containers
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.106389    2581 manager.go:354] Recovery completed
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.109371    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.109370    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-218.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.111269    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.113538    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.113589    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.113610    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.114670    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.114684    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.114702    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.114718    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.115471    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.115496    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.115508    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:21:28.465331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.115874    2581 policy_none.go:49] "None policy: Start"
Jul 01 19:21:28.466248 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.115889    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 01 19:21:28.466248 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.115900    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 01 19:21:28.466248 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.116335    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-218.ec2.internal.18be40be3216163b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-218.ec2.internal,UID:ip-10-0-138-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-218.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-218.ec2.internal,},FirstTimestamp:2026-07-01 19:21:28.113567291 +0000 UTC m=+0.383315419,LastTimestamp:2026-07-01 19:21:28.113567291 +0000 UTC m=+0.383315419,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-218.ec2.internal,}"
Jul 01 19:21:28.466248 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.117119    2581 policy_none.go:47] "Start"
Jul 01 19:21:28.466248 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.134521    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-218.ec2.internal.18be40be32168f06  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-218.ec2.internal,UID:ip-10-0-138-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-218.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-218.ec2.internal,},FirstTimestamp:2026-07-01 19:21:28.113598214 +0000 UTC m=+0.383346333,LastTimestamp:2026-07-01 19:21:28.113598214 +0000 UTC m=+0.383346333,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-218.ec2.internal,}"
Jul 01 19:21:28.466248 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.139020    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-218.ec2.internal.18be40be3216d9ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-218.ec2.internal,UID:ip-10-0-138-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-218.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-218.ec2.internal,},FirstTimestamp:2026-07-01 19:21:28.113617358 +0000 UTC m=+0.383365482,LastTimestamp:2026-07-01 19:21:28.113617358 +0000 UTC m=+0.383365482,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-218.ec2.internal,}"
Jul 01 19:21:28.466248 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.146854    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-218.ec2.internal.18be40be3216163b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-218.ec2.internal.18be40be3216163b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-218.ec2.internal,UID:ip-10-0-138-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-218.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-218.ec2.internal,},FirstTimestamp:2026-07-01 19:21:28.113567291 +0000 UTC m=+0.383315419,LastTimestamp:2026-07-01 19:21:28.115482831 +0000 UTC m=+0.385230949,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-218.ec2.internal,}"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.151662    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-218.ec2.internal.18be40be32168f06\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-218.ec2.internal.18be40be32168f06  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-218.ec2.internal,UID:ip-10-0-138-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-218.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-218.ec2.internal,},FirstTimestamp:2026-07-01 19:21:28.113598214 +0000 UTC m=+0.383346333,LastTimestamp:2026-07-01 19:21:28.115501787 +0000 UTC m=+0.385249906,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-218.ec2.internal,}"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.154633    2581 manager.go:337] "Starting Device Plugin manager"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.154666    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.154680    2581 server.go:87] "Starting device plugin registration server"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.154933    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.154945    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.155014    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.155090    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.155099    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.155682    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.155722    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-218.ec2.internal\" not found"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.160834    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-218.ec2.internal.18be40be3216d9ce\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-218.ec2.internal.18be40be3216d9ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-218.ec2.internal,UID:ip-10-0-138-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-218.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-218.ec2.internal,},FirstTimestamp:2026-07-01 19:21:28.113617358 +0000 UTC m=+0.383365482,LastTimestamp:2026-07-01 19:21:28.115512482 +0000 UTC m=+0.385260602,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-218.ec2.internal,}"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.171151    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.174534    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.174573    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.174598    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.174639    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.178570    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4rc4f"
Jul 01 19:21:28.661486 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.181164    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-218.ec2.internal.18be40be351cf141  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-218.ec2.internal,UID:ip-10-0-138-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-138-218.ec2.internal,},FirstTimestamp:2026-07-01 19:21:28.164348225 +0000 UTC m=+0.434096331,LastTimestamp:2026-07-01 19:21:28.164348225 +0000 UTC m=+0.434096331,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-218.ec2.internal,}"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.181231    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.190345    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4rc4f"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.256021    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.257042    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.257072    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.257082    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.257104    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.261790    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-218.ec2.internal.18be40be3216163b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-218.ec2.internal.18be40be3216163b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-218.ec2.internal,UID:ip-10-0-138-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-218.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-218.ec2.internal,},FirstTimestamp:2026-07-01 19:21:28.113567291 +0000 UTC m=+0.383315419,LastTimestamp:2026-07-01 19:21:28.257058239 +0000 UTC m=+0.526806358,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-218.ec2.internal,}"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.274789    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-218.ec2.internal"]
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.274849    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.276095    2581 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-138-218.ec2.internal\" not found" node="ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.278765    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.278798    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.278810    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.280214    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.280277    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.280303    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.280922    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.280933    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.280974    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.280986    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.280943    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.281063    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.282262    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.662896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.282284    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.282975    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.282998    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.283007    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.298814    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4752b2898576bdb47b1cf0f1601ae0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal\" (UID: \"d4752b2898576bdb47b1cf0f1601ae0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.298839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4752b2898576bdb47b1cf0f1601ae0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal\" (UID: \"d4752b2898576bdb47b1cf0f1601ae0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.298860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5bf0f4d1f6b89cb065876b678658105c-config\") pod \"kube-apiserver-proxy-ip-10-0-138-218.ec2.internal\" (UID: \"5bf0f4d1f6b89cb065876b678658105c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.313746    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-218.ec2.internal\" not found" node="ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.318120    2581 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-218.ec2.internal\" not found" node="ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.318194    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-218.ec2.internal\" not found" node="ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.399424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4752b2898576bdb47b1cf0f1601ae0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal\" (UID: \"d4752b2898576bdb47b1cf0f1601ae0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.399456    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4752b2898576bdb47b1cf0f1601ae0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal\" (UID: \"d4752b2898576bdb47b1cf0f1601ae0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.399482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5bf0f4d1f6b89cb065876b678658105c-config\") pod \"kube-apiserver-proxy-ip-10-0-138-218.ec2.internal\" (UID: \"5bf0f4d1f6b89cb065876b678658105c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.399521    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5bf0f4d1f6b89cb065876b678658105c-config\") pod \"kube-apiserver-proxy-ip-10-0-138-218.ec2.internal\" (UID: \"5bf0f4d1f6b89cb065876b678658105c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.399529    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4752b2898576bdb47b1cf0f1601ae0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal\" (UID: \"d4752b2898576bdb47b1cf0f1601ae0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.399537    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4752b2898576bdb47b1cf0f1601ae0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal\" (UID: \"d4752b2898576bdb47b1cf0f1601ae0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.476326    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.477392    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 19:21:28.663770 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.477425    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 19:21:28.664356 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.477437    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeHasSufficientPID"
Jul 01 19:21:28.664356 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.477471    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.664356 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.486561    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.664356 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.486584    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-218.ec2.internal\": node \"ip-10-0-138-218.ec2.internal\" not found"
Jul 01 19:21:28.664356 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.556651    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-218.ec2.internal\" not found"
Jul 01 19:21:28.664356 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.617169    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.664356 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:28.623251    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:28.664356 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.657272    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-218.ec2.internal\" not found"
Jul 01 19:21:28.898171 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.757773    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-218.ec2.internal\" not found"
Jul 01 19:21:28.898171 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.858273    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-218.ec2.internal\" not found"
Jul 01 19:21:29.127845 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:28.958911    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-218.ec2.internal\" not found"
Jul 01 19:21:29.127845 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:29.031392    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 01 19:21:29.127845 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:29.059694    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-218.ec2.internal\" not found"
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:29.160142    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-218.ec2.internal\" not found"
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:29.176040    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:29.195679    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 19:16:29 +0000 UTC" deadline="2028-01-15 21:35:17.187900073 +0000 UTC"
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:29.195710    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13514h13m47.992193413s"
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:29.203045    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf0f4d1f6b89cb065876b678658105c.slice/crio-48c85ecbd65179e576f9440a5a559c52369cdbc0583a9fa655e46b95c9cecae7 WatchSource:0}: Error finding container 48c85ecbd65179e576f9440a5a559c52369cdbc0583a9fa655e46b95c9cecae7: Status 404 returned error can't find the container with id 48c85ecbd65179e576f9440a5a559c52369cdbc0583a9fa655e46b95c9cecae7
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:29.203247    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4752b2898576bdb47b1cf0f1601ae0e.slice/crio-45329afb27dccd724e9785678b3710a083f27ffc2e02faba5562f9ea586274aa WatchSource:0}: Error finding container 45329afb27dccd724e9785678b3710a083f27ffc2e02faba5562f9ea586274aa: Status 404 returned error can't find the container with id 45329afb27dccd724e9785678b3710a083f27ffc2e02faba5562f9ea586274aa
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:29.207340    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:29.221444    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:29.227890    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:29.297659    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:29.311365    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:29.312464    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-218.ec2.internal"
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:29.324456    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 19:21:29.378993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:29.378118    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 19:21:31.108358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.108277    2581 apiserver.go:52] "Watching apiserver"
Jul 01 19:21:31.120023 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.119994    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 01 19:21:31.120456 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.120428    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-vw6c6","openshift-multus/network-metrics-daemon-lw425","openshift-network-diagnostics/network-check-target-595cz","openshift-network-operator/iptables-alerter-mdvx6","openshift-ovn-kubernetes/ovnkube-node-mhs48","kube-system/kube-apiserver-proxy-ip-10-0-138-218.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng","openshift-cluster-node-tuning-operator/tuned-2t8qd","openshift-dns/node-resolver-hmg26","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal","openshift-multus/multus-nftfz","kube-system/global-pull-secret-syncer-n9vnj","kube-system/konnectivity-agent-z2d87","openshift-image-registry/node-ca-rdn6x"]
Jul 01 19:21:31.122494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.122374    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.123640 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.123502    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z2d87"
Jul 01 19:21:31.124703 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.124682    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:31.124797 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.124761    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:31.126271 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.125844    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mdvx6"
Jul 01 19:21:31.127505 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.127384    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.128835 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.128732    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rdn6x"
Jul 01 19:21:31.128835 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.128757    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 01 19:21:31.129862 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.129840    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mtdhx\""
Jul 01 19:21:31.130611 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.130159    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 01 19:21:31.130611 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.130170    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 01 19:21:31.130611 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.130483    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 01 19:21:31.131149 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.130837    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cn587\""
Jul 01 19:21:31.131245 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.131221    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 01 19:21:31.131857 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.131836    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 01 19:21:31.131949 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.131868    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9jt9m\""
Jul 01 19:21:31.132935 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.132775    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 01 19:21:31.134401 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.133082    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 01 19:21:31.134401 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.133368    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 01 19:21:31.134401 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.133408    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 01 19:21:31.134401 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.133674    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 01 19:21:31.134401 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.133700    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 01 19:21:31.134401 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.133986    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 01 19:21:31.134401 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.134148    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 01 19:21:31.134401 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.134192    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 01 19:21:31.134401 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.134368    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fc8ph\""
Jul 01 19:21:31.135048 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.135023    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 01 19:21:31.136921 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.136452    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hmg26"
Jul 01 19:21:31.139506 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.139488    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 01 19:21:31.139811 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.139796    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j2vqf\""
Jul 01 19:21:31.140003 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.139984    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 01 19:21:31.140101 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.140082    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.140670 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.140650    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:31.140748 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.140712    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:31.141431 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.141301    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 01 19:21:31.142070 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.141917    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.143211 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143140    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-kubelet\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.143295 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-run-systemd\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.143295 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143254    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j44lw\" (UniqueName: \"kubernetes.io/projected/00551914-29ac-4574-a3f3-b5e78dd6dca4-kube-api-access-j44lw\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.143295 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143280    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-socket-dir\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.143439 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143311    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9d38693f-5c9b-4e27-8894-7b190acc75db-iptables-alerter-script\") pod \"iptables-alerter-mdvx6\" (UID: \"9d38693f-5c9b-4e27-8894-7b190acc75db\") " pod="openshift-network-operator/iptables-alerter-mdvx6"
Jul 01 19:21:31.143439 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143390    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-run-netns\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.143439 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-cni-netd\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.143600 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00551914-29ac-4574-a3f3-b5e78dd6dca4-ovnkube-config\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.143600 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143485    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a8e8c5-fc6c-49cf-8726-3b0cf31d8464-hosts-file\") pod \"node-resolver-hmg26\" (UID: \"78a8e8c5-fc6c-49cf-8726-3b0cf31d8464\") " pod="openshift-dns/node-resolver-hmg26"
Jul 01 19:21:31.143600 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143517    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.143600 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-registration-dir\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.143600 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.143835 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143621    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00551914-29ac-4574-a3f3-b5e78dd6dca4-env-overrides\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.143835 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143646    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.143835 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00551914-29ac-4574-a3f3-b5e78dd6dca4-ovn-node-metrics-cert\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.143835 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143722    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d38693f-5c9b-4e27-8894-7b190acc75db-host-slash\") pod \"iptables-alerter-mdvx6\" (UID: \"9d38693f-5c9b-4e27-8894-7b190acc75db\") " pod="openshift-network-operator/iptables-alerter-mdvx6"
Jul 01 19:21:31.143835 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143746    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-slash\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.143835 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-node-log\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.143835 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143790    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c22da25a-46e6-4260-886b-98c2a65d9bbf-konnectivity-ca\") pod \"konnectivity-agent-z2d87\" (UID: \"c22da25a-46e6-4260-886b-98c2a65d9bbf\") " pod="kube-system/konnectivity-agent-z2d87"
Jul 01 19:21:31.143835 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143812    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-systemd-units\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.143835 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-var-lib-openvswitch\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.144226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143858    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-run-ovn-kubernetes\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.144226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143882    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-cni-bin\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.144226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143906    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8556e73-373f-49ed-b16d-a1c51b253357-host\") pod \"node-ca-rdn6x\" (UID: \"b8556e73-373f-49ed-b16d-a1c51b253357\") " pod="openshift-image-registry/node-ca-rdn6x"
Jul 01 19:21:31.144226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143929    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b8556e73-373f-49ed-b16d-a1c51b253357-serviceca\") pod \"node-ca-rdn6x\" (UID: \"b8556e73-373f-49ed-b16d-a1c51b253357\") " pod="openshift-image-registry/node-ca-rdn6x"
Jul 01 19:21:31.144226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd9fc\" (UniqueName: \"kubernetes.io/projected/b8556e73-373f-49ed-b16d-a1c51b253357-kube-api-access-dd9fc\") pod \"node-ca-rdn6x\" (UID: \"b8556e73-373f-49ed-b16d-a1c51b253357\") " pod="openshift-image-registry/node-ca-rdn6x"
Jul 01 19:21:31.144226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.143977    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-etc-openvswitch\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.144226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144001    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-run-openvswitch\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.144226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-log-socket\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.144226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144049    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00551914-29ac-4574-a3f3-b5e78dd6dca4-ovnkube-script-lib\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.144226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144111    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a8e8c5-fc6c-49cf-8726-3b0cf31d8464-tmp-dir\") pod \"node-resolver-hmg26\" (UID: \"78a8e8c5-fc6c-49cf-8726-3b0cf31d8464\") " pod="openshift-dns/node-resolver-hmg26"
Jul 01 19:21:31.144226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9xpp\" (UniqueName: \"kubernetes.io/projected/78a8e8c5-fc6c-49cf-8726-3b0cf31d8464-kube-api-access-t9xpp\") pod \"node-resolver-hmg26\" (UID: \"78a8e8c5-fc6c-49cf-8726-3b0cf31d8464\") " pod="openshift-dns/node-resolver-hmg26"
Jul 01 19:21:31.144742 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144240    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-etc-selinux\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.144742 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144266    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-sys-fs\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.144742 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144289    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btldv\" (UniqueName: \"kubernetes.io/projected/9d38693f-5c9b-4e27-8894-7b190acc75db-kube-api-access-btldv\") pod \"iptables-alerter-mdvx6\" (UID: \"9d38693f-5c9b-4e27-8894-7b190acc75db\") " pod="openshift-network-operator/iptables-alerter-mdvx6"
Jul 01 19:21:31.144742 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144311    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c22da25a-46e6-4260-886b-98c2a65d9bbf-agent-certs\") pod \"konnectivity-agent-z2d87\" (UID: \"c22da25a-46e6-4260-886b-98c2a65d9bbf\") " pod="kube-system/konnectivity-agent-z2d87"
Jul 01 19:21:31.144742 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-run-ovn\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.144742 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144371    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-device-dir\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.144742 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144395    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs72s\" (UniqueName: \"kubernetes.io/projected/a709ebfa-f726-4206-acdc-2e39ba823a35-kube-api-access-hs72s\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.144742 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.144419    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvqrk\" (UniqueName: \"kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk\") pod \"network-check-target-595cz\" (UID: \"e93ed59e-2406-4e87-a299-26c95b66a0d2\") " pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:31.145385 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.145273    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:31.145385 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.145329    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:31.147360 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.147341    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 01 19:21:31.147487 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.147471    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 01 19:21:31.147487 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.147480    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 01 19:21:31.147566 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.147550    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 01 19:21:31.147769 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.147755    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 01 19:21:31.148128 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.148110    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 01 19:21:31.148269 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.148238    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 01 19:21:31.148466 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.148440    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 01 19:21:31.148550 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.148464    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jxnzp\""
Jul 01 19:21:31.148742 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.148677    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fwn8b\""
Jul 01 19:21:31.148819 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.148771    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-d2gr7\""
Jul 01 19:21:31.148912 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.148897    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t8cgn\""
Jul 01 19:21:31.149006 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.148972    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 01 19:21:31.156909 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.156822    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 19:21:31.209762 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.209727    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wm645"
Jul 01 19:21:31.216735 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.216656    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-218.ec2.internal" event={"ID":"5bf0f4d1f6b89cb065876b678658105c","Type":"ContainerStarted","Data":"48c85ecbd65179e576f9440a5a559c52369cdbc0583a9fa655e46b95c9cecae7"}
Jul 01 19:21:31.217983 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.217935    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal" event={"ID":"d4752b2898576bdb47b1cf0f1601ae0e","Type":"ContainerStarted","Data":"45329afb27dccd724e9785678b3710a083f27ffc2e02faba5562f9ea586274aa"}
Jul 01 19:21:31.222416 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.222390    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wm645"
Jul 01 19:21:31.233052 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.233031    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 01 19:21:31.244603 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244551    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-cni-netd\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.244603 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00551914-29ac-4574-a3f3-b5e78dd6dca4-ovnkube-config\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.244784 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244627    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.244784 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-registration-dir\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.244784 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-etc-selinux\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.244784 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-sys-fs\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.244784 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-slash\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.244784 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-node-log\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.244784 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244749    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-cni-bin\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-multus-socket-dir-parent\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-var-lib-cni-multus\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244844    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b2f63e26-b149-4c9e-b712-69e1add49e6c-multus-daemon-config\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-modprobe-d\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-lib-modules\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244904    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c22da25a-46e6-4260-886b-98c2a65d9bbf-konnectivity-ca\") pod \"konnectivity-agent-z2d87\" (UID: \"c22da25a-46e6-4260-886b-98c2a65d9bbf\") " pod="kube-system/konnectivity-agent-z2d87"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244926    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-run-ovn-kubernetes\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244950    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a8e8c5-fc6c-49cf-8726-3b0cf31d8464-tmp-dir\") pod \"node-resolver-hmg26\" (UID: \"78a8e8c5-fc6c-49cf-8726-3b0cf31d8464\") " pod="openshift-dns/node-resolver-hmg26"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244967    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-os-release\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244983    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/314e23aa-a01f-48a2-9b68-ed39d403485c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.244997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-run\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-etc-openvswitch\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245029    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00551914-29ac-4574-a3f3-b5e78dd6dca4-ovnkube-script-lib\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.245141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-var-lib-kubelet\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245065    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-var-lib-kubelet\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-sys\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245072    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-cni-netd\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/617fc997-a13b-4f86-96ea-378a55762874-tmp\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245121    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-var-lib-cni-bin\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-run-ovn\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvqrk\" (UniqueName: \"kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk\") pod \"network-check-target-595cz\" (UID: \"e93ed59e-2406-4e87-a299-26c95b66a0d2\") " pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245200    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-system-cni-dir\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-systemd\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245246    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-kubelet\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245269    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-multus-cni-dir\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245291    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-run-multus-certs\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245313    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-sysconfig\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.245798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245354    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-sysctl-d\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245387    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-socket-dir\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245413    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb88v\" (UniqueName: \"kubernetes.io/projected/617fc997-a13b-4f86-96ea-378a55762874-kube-api-access-nb88v\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245437    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2f63e26-b149-4c9e-b712-69e1add49e6c-cni-binary-copy\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245456    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-run-netns\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a8e8c5-fc6c-49cf-8726-3b0cf31d8464-hosts-file\") pod \"node-resolver-hmg26\" (UID: \"78a8e8c5-fc6c-49cf-8726-3b0cf31d8464\") " pod="openshift-dns/node-resolver-hmg26"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245503    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245526    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlsss\" (UniqueName: \"kubernetes.io/projected/161462f4-ad95-4e5a-ab54-148c05190926-kube-api-access-hlsss\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245553    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245590    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00551914-29ac-4574-a3f3-b5e78dd6dca4-env-overrides\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00551914-29ac-4574-a3f3-b5e78dd6dca4-ovn-node-metrics-cert\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d38693f-5c9b-4e27-8894-7b190acc75db-host-slash\") pod \"iptables-alerter-mdvx6\" (UID: \"9d38693f-5c9b-4e27-8894-7b190acc75db\") " pod="openshift-network-operator/iptables-alerter-mdvx6"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8556e73-373f-49ed-b16d-a1c51b253357-host\") pod \"node-ca-rdn6x\" (UID: \"b8556e73-373f-49ed-b16d-a1c51b253357\") " pod="openshift-image-registry/node-ca-rdn6x"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245693    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b8556e73-373f-49ed-b16d-a1c51b253357-serviceca\") pod \"node-ca-rdn6x\" (UID: \"b8556e73-373f-49ed-b16d-a1c51b253357\") " pod="openshift-image-registry/node-ca-rdn6x"
Jul 01 19:21:31.246421 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-systemd-units\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245724    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00551914-29ac-4574-a3f3-b5e78dd6dca4-ovnkube-config\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245749    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-sys-fs\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245810    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245864    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-var-lib-openvswitch\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245901    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-registration-dir\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245933    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-kubelet\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246014    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-slash\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246032    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246092    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-node-log\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246144    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-cni-bin\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.245738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-var-lib-openvswitch\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd9fc\" (UniqueName: \"kubernetes.io/projected/b8556e73-373f-49ed-b16d-a1c51b253357-kube-api-access-dd9fc\") pod \"node-ca-rdn6x\" (UID: \"b8556e73-373f-49ed-b16d-a1c51b253357\") " pod="openshift-image-registry/node-ca-rdn6x"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246240    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9xpp\" (UniqueName: \"kubernetes.io/projected/78a8e8c5-fc6c-49cf-8726-3b0cf31d8464-kube-api-access-t9xpp\") pod \"node-resolver-hmg26\" (UID: \"78a8e8c5-fc6c-49cf-8726-3b0cf31d8464\") " pod="openshift-dns/node-resolver-hmg26"
Jul 01 19:21:31.246994 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl9rn\" (UniqueName: \"kubernetes.io/projected/b2f63e26-b149-4c9e-b712-69e1add49e6c-kube-api-access-cl9rn\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.247480 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246318    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/85fa991c-3b94-4f96-ab9b-608b29ad2b20-kubelet-config\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:31.247480 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246357    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:31.247480 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246401    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-kubernetes\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.247480 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246480    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-run-openvswitch\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.247480 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-log-socket\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.247480 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246535    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-run-netns\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.247480 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-host\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.247480 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btldv\" (UniqueName: \"kubernetes.io/projected/9d38693f-5c9b-4e27-8894-7b190acc75db-kube-api-access-btldv\") pod \"iptables-alerter-mdvx6\" (UID: \"9d38693f-5c9b-4e27-8894-7b190acc75db\") " pod="openshift-network-operator/iptables-alerter-mdvx6"
Jul 01 19:21:31.247480 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/314e23aa-a01f-48a2-9b68-ed39d403485c-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.247915 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.246675    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwtqw\" (UniqueName: \"kubernetes.io/projected/314e23aa-a01f-48a2-9b68-ed39d403485c-kube-api-access-lwtqw\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.247972 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.247942    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-run-netns\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.248086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.248066    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a8e8c5-fc6c-49cf-8726-3b0cf31d8464-hosts-file\") pod \"node-resolver-hmg26\" (UID: \"78a8e8c5-fc6c-49cf-8726-3b0cf31d8464\") " pod="openshift-dns/node-resolver-hmg26"
Jul 01 19:21:31.248143 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.248122    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-hostroot\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.248190 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.248165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c22da25a-46e6-4260-886b-98c2a65d9bbf-agent-certs\") pod \"konnectivity-agent-z2d87\" (UID: \"c22da25a-46e6-4260-886b-98c2a65d9bbf\") " pod="kube-system/konnectivity-agent-z2d87"
Jul 01 19:21:31.248241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.248212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-device-dir\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.248345 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.248248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs72s\" (UniqueName: \"kubernetes.io/projected/a709ebfa-f726-4206-acdc-2e39ba823a35-kube-api-access-hs72s\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.248447 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.248357    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 01 19:21:31.248447 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.248438    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00551914-29ac-4574-a3f3-b5e78dd6dca4-ovnkube-script-lib\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.248558 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.248516    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-log-socket\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.248996 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.248956    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-run-ovn\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.249167 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.249135    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d38693f-5c9b-4e27-8894-7b190acc75db-host-slash\") pod \"iptables-alerter-mdvx6\" (UID: \"9d38693f-5c9b-4e27-8894-7b190acc75db\") " pod="openshift-network-operator/iptables-alerter-mdvx6"
Jul 01 19:21:31.249265 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.249243    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c22da25a-46e6-4260-886b-98c2a65d9bbf-konnectivity-ca\") pod \"konnectivity-agent-z2d87\" (UID: \"c22da25a-46e6-4260-886b-98c2a65d9bbf\") " pod="kube-system/konnectivity-agent-z2d87"
Jul 01 19:21:31.249526 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.249510    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-etc-selinux\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.249668 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.249650    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-systemd-units\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.249837 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.249801    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00551914-29ac-4574-a3f3-b5e78dd6dca4-env-overrides\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.250076 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.250039    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-host-run-ovn-kubernetes\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.250195 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.250179    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-socket-dir\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.250500 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.250483    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a8e8c5-fc6c-49cf-8726-3b0cf31d8464-tmp-dir\") pod \"node-resolver-hmg26\" (UID: \"78a8e8c5-fc6c-49cf-8726-3b0cf31d8464\") " pod="openshift-dns/node-resolver-hmg26"
Jul 01 19:21:31.251897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251300    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b8556e73-373f-49ed-b16d-a1c51b253357-serviceca\") pod \"node-ca-rdn6x\" (UID: \"b8556e73-373f-49ed-b16d-a1c51b253357\") " pod="openshift-image-registry/node-ca-rdn6x"
Jul 01 19:21:31.251897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251368    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a709ebfa-f726-4206-acdc-2e39ba823a35-device-dir\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.251897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251402    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-cnibin\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.251897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251433    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-etc-kubernetes\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.251897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251519    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/85fa991c-3b94-4f96-ab9b-608b29ad2b20-dbus\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:31.251897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-run-systemd\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.251897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/314e23aa-a01f-48a2-9b68-ed39d403485c-system-cni-dir\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.251897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/314e23aa-a01f-48a2-9b68-ed39d403485c-cnibin\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.251897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251741    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/314e23aa-a01f-48a2-9b68-ed39d403485c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.251897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j44lw\" (UniqueName: \"kubernetes.io/projected/00551914-29ac-4574-a3f3-b5e78dd6dca4-kube-api-access-j44lw\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.251897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251815    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-run-k8s-cni-cncf-io\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.251897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251847    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/314e23aa-a01f-48a2-9b68-ed39d403485c-os-release\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.252564 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251882    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-run-openvswitch\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.252564 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.251912    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-etc-openvswitch\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.252564 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.252034    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00551914-29ac-4574-a3f3-b5e78dd6dca4-run-systemd\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.252564 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.252338    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8556e73-373f-49ed-b16d-a1c51b253357-host\") pod \"node-ca-rdn6x\" (UID: \"b8556e73-373f-49ed-b16d-a1c51b253357\") " pod="openshift-image-registry/node-ca-rdn6x"
Jul 01 19:21:31.252564 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.252382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/314e23aa-a01f-48a2-9b68-ed39d403485c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.252564 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.252470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-sysctl-conf\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.252564 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.252507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/617fc997-a13b-4f86-96ea-378a55762874-etc-tuned\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.252564 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.252550    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-multus-conf-dir\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.252967 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.252682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9d38693f-5c9b-4e27-8894-7b190acc75db-iptables-alerter-script\") pod \"iptables-alerter-mdvx6\" (UID: \"9d38693f-5c9b-4e27-8894-7b190acc75db\") " pod="openshift-network-operator/iptables-alerter-mdvx6"
Jul 01 19:21:31.253378 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.253355    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9d38693f-5c9b-4e27-8894-7b190acc75db-iptables-alerter-script\") pod \"iptables-alerter-mdvx6\" (UID: \"9d38693f-5c9b-4e27-8894-7b190acc75db\") " pod="openshift-network-operator/iptables-alerter-mdvx6"
Jul 01 19:21:31.253968 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.253560    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00551914-29ac-4574-a3f3-b5e78dd6dca4-ovn-node-metrics-cert\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.254997 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.254941    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c22da25a-46e6-4260-886b-98c2a65d9bbf-agent-certs\") pod \"konnectivity-agent-z2d87\" (UID: \"c22da25a-46e6-4260-886b-98c2a65d9bbf\") " pod="kube-system/konnectivity-agent-z2d87"
Jul 01 19:21:31.256366 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.256319    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:21:31.256366 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.256342    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:21:31.256366 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.256355    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kvqrk for pod openshift-network-diagnostics/network-check-target-595cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:21:31.256655 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.256429    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk podName:e93ed59e-2406-4e87-a299-26c95b66a0d2 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:31.756407833 +0000 UTC m=+2.991533142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kvqrk" (UniqueName: "kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk") pod "network-check-target-595cz" (UID: "e93ed59e-2406-4e87-a299-26c95b66a0d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:21:31.268898 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.268869    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btldv\" (UniqueName: \"kubernetes.io/projected/9d38693f-5c9b-4e27-8894-7b190acc75db-kube-api-access-btldv\") pod \"iptables-alerter-mdvx6\" (UID: \"9d38693f-5c9b-4e27-8894-7b190acc75db\") " pod="openshift-network-operator/iptables-alerter-mdvx6"
Jul 01 19:21:31.269481 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.269437    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs72s\" (UniqueName: \"kubernetes.io/projected/a709ebfa-f726-4206-acdc-2e39ba823a35-kube-api-access-hs72s\") pod \"aws-ebs-csi-driver-node-tbvng\" (UID: \"a709ebfa-f726-4206-acdc-2e39ba823a35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.271080 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.271058    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9xpp\" (UniqueName: \"kubernetes.io/projected/78a8e8c5-fc6c-49cf-8726-3b0cf31d8464-kube-api-access-t9xpp\") pod \"node-resolver-hmg26\" (UID: \"78a8e8c5-fc6c-49cf-8726-3b0cf31d8464\") " pod="openshift-dns/node-resolver-hmg26"
Jul 01 19:21:31.271080 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.271074    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j44lw\" (UniqueName: \"kubernetes.io/projected/00551914-29ac-4574-a3f3-b5e78dd6dca4-kube-api-access-j44lw\") pod \"ovnkube-node-mhs48\" (UID: \"00551914-29ac-4574-a3f3-b5e78dd6dca4\") " pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.271465 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.271447    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd9fc\" (UniqueName: \"kubernetes.io/projected/b8556e73-373f-49ed-b16d-a1c51b253357-kube-api-access-dd9fc\") pod \"node-ca-rdn6x\" (UID: \"b8556e73-373f-49ed-b16d-a1c51b253357\") " pod="openshift-image-registry/node-ca-rdn6x"
Jul 01 19:21:31.353633 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-run\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.353633 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-var-lib-kubelet\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.353881 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-var-lib-kubelet\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.353881 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353727    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-var-lib-kubelet\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.353881 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353743    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-var-lib-kubelet\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.353881 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353779    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-sys\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.353881 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/617fc997-a13b-4f86-96ea-378a55762874-tmp\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.353881 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353813    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-sys\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.353881 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-var-lib-cni-bin\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.353881 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353782    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-run\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353869    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-system-cni-dir\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353886    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-var-lib-cni-bin\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-systemd\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353943    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-multus-cni-dir\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353960    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-systemd\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353968    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-run-multus-certs\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.353992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-sysconfig\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-sysctl-d\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354053    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-system-cni-dir\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb88v\" (UniqueName: \"kubernetes.io/projected/617fc997-a13b-4f86-96ea-378a55762874-kube-api-access-nb88v\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354090    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-multus-cni-dir\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354093    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2f63e26-b149-4c9e-b712-69e1add49e6c-cni-binary-copy\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354159    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlsss\" (UniqueName: \"kubernetes.io/projected/161462f4-ad95-4e5a-ab54-148c05190926-kube-api-access-hlsss\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:31.354241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354197    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl9rn\" (UniqueName: \"kubernetes.io/projected/b2f63e26-b149-4c9e-b712-69e1add49e6c-kube-api-access-cl9rn\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354224    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/85fa991c-3b94-4f96-ab9b-608b29ad2b20-kubelet-config\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-kubernetes\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.354315    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354325    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-run-netns\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354337    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-run-multus-certs\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354350    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-host\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/314e23aa-a01f-48a2-9b68-ed39d403485c-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354390    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-sysconfig\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.354399    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs podName:161462f4-ad95-4e5a-ab54-148c05190926 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:31.854377871 +0000 UTC m=+3.089503185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs") pod "network-metrics-daemon-lw425" (UID: "161462f4-ad95-4e5a-ab54-148c05190926") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354498    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-sysctl-d\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.354590    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.354653    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret podName:85fa991c-3b94-4f96-ab9b-608b29ad2b20 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:31.854636437 +0000 UTC m=+3.089761737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret") pod "global-pull-secret-syncer-n9vnj" (UID: "85fa991c-3b94-4f96-ab9b-608b29ad2b20") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:21:31.355494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354654    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-run-netns\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwtqw\" (UniqueName: \"kubernetes.io/projected/314e23aa-a01f-48a2-9b68-ed39d403485c-kube-api-access-lwtqw\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354681    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-host\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354711    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-hostroot\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-cnibin\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354774    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/85fa991c-3b94-4f96-ab9b-608b29ad2b20-kubelet-config\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-etc-kubernetes\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354825    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-etc-kubernetes\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354850    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/85fa991c-3b94-4f96-ab9b-608b29ad2b20-dbus\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354717    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-kubernetes\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354879    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/314e23aa-a01f-48a2-9b68-ed39d403485c-system-cni-dir\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354904    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/314e23aa-a01f-48a2-9b68-ed39d403485c-cnibin\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354923    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-hostroot\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354939    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/314e23aa-a01f-48a2-9b68-ed39d403485c-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354716    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2f63e26-b149-4c9e-b712-69e1add49e6c-cni-binary-copy\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.356182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354963    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-cnibin\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354963    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/314e23aa-a01f-48a2-9b68-ed39d403485c-cnibin\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354978    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/85fa991c-3b94-4f96-ab9b-608b29ad2b20-dbus\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.354983    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/314e23aa-a01f-48a2-9b68-ed39d403485c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355003    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/314e23aa-a01f-48a2-9b68-ed39d403485c-system-cni-dir\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355014    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-run-k8s-cni-cncf-io\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355039    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/314e23aa-a01f-48a2-9b68-ed39d403485c-os-release\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355044    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-run-k8s-cni-cncf-io\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355062    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/314e23aa-a01f-48a2-9b68-ed39d403485c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-sysctl-conf\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355094    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/314e23aa-a01f-48a2-9b68-ed39d403485c-os-release\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355105    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/617fc997-a13b-4f86-96ea-378a55762874-etc-tuned\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355149    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-multus-conf-dir\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-multus-socket-dir-parent\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.356803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355218    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-var-lib-cni-multus\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b2f63e26-b149-4c9e-b712-69e1add49e6c-multus-daemon-config\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355245    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-sysctl-conf\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-modprobe-d\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-lib-modules\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355331    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-os-release\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/314e23aa-a01f-48a2-9b68-ed39d403485c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355440    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/314e23aa-a01f-48a2-9b68-ed39d403485c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355510    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/314e23aa-a01f-48a2-9b68-ed39d403485c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355534    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-etc-modprobe-d\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355593    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-host-var-lib-cni-multus\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355595    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/314e23aa-a01f-48a2-9b68-ed39d403485c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355594    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-os-release\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355625    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-multus-socket-dir-parent\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355636    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2f63e26-b149-4c9e-b712-69e1add49e6c-multus-conf-dir\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.357324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.355703    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/617fc997-a13b-4f86-96ea-378a55762874-lib-modules\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.357843 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.356003    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b2f63e26-b149-4c9e-b712-69e1add49e6c-multus-daemon-config\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.357843 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.356917    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/617fc997-a13b-4f86-96ea-378a55762874-tmp\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.357843 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.357420    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/617fc997-a13b-4f86-96ea-378a55762874-etc-tuned\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.367258 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.367202    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb88v\" (UniqueName: \"kubernetes.io/projected/617fc997-a13b-4f86-96ea-378a55762874-kube-api-access-nb88v\") pod \"tuned-2t8qd\" (UID: \"617fc997-a13b-4f86-96ea-378a55762874\") " pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.373875 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.373827    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlsss\" (UniqueName: \"kubernetes.io/projected/161462f4-ad95-4e5a-ab54-148c05190926-kube-api-access-hlsss\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:31.374002 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.373920    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl9rn\" (UniqueName: \"kubernetes.io/projected/b2f63e26-b149-4c9e-b712-69e1add49e6c-kube-api-access-cl9rn\") pod \"multus-nftfz\" (UID: \"b2f63e26-b149-4c9e-b712-69e1add49e6c\") " pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.375866 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.375846    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwtqw\" (UniqueName: \"kubernetes.io/projected/314e23aa-a01f-48a2-9b68-ed39d403485c-kube-api-access-lwtqw\") pod \"multus-additional-cni-plugins-vw6c6\" (UID: \"314e23aa-a01f-48a2-9b68-ed39d403485c\") " pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.434430 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.434392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng"
Jul 01 19:21:31.448969 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.448941    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z2d87"
Jul 01 19:21:31.457971 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.457943    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mdvx6"
Jul 01 19:21:31.463823 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.463801    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:31.471546 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.471516    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rdn6x"
Jul 01 19:21:31.479266 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.479236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hmg26"
Jul 01 19:21:31.487406 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.487382    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vw6c6"
Jul 01 19:21:31.495137 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.495112    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2t8qd"
Jul 01 19:21:31.502277 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.502248    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nftfz"
Jul 01 19:21:31.758331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.758231    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvqrk\" (UniqueName: \"kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk\") pod \"network-check-target-595cz\" (UID: \"e93ed59e-2406-4e87-a299-26c95b66a0d2\") " pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:31.758501 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.758433    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:21:31.758501 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.758460    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:21:31.758501 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.758472    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kvqrk for pod openshift-network-diagnostics/network-check-target-595cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:21:31.758685 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.758544    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk podName:e93ed59e-2406-4e87-a299-26c95b66a0d2 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:32.758522661 +0000 UTC m=+3.993647965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvqrk" (UniqueName: "kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk") pod "network-check-target-595cz" (UID: "e93ed59e-2406-4e87-a299-26c95b66a0d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:21:31.858774 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.858727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:31.858953 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:31.858823    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:31.858953 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.858885    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:21:31.858953 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.858921    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:21:31.859107 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.858959    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret podName:85fa991c-3b94-4f96-ab9b-608b29ad2b20 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:32.858940752 +0000 UTC m=+4.094066045 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret") pod "global-pull-secret-syncer-n9vnj" (UID: "85fa991c-3b94-4f96-ab9b-608b29ad2b20") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:21:31.859107 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:31.858982    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs podName:161462f4-ad95-4e5a-ab54-148c05190926 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:32.858971581 +0000 UTC m=+4.094096871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs") pod "network-metrics-daemon-lw425" (UID: "161462f4-ad95-4e5a-ab54-148c05190926") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:21:32.224639 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:32.224589    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 19:16:31 +0000 UTC" deadline="2027-11-26 04:40:52.441821673 +0000 UTC"
Jul 01 19:21:32.224748 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:32.224640    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12297h19m20.217185477s"
Jul 01 19:21:32.225256 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:32.225233    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod314e23aa_a01f_48a2_9b68_ed39d403485c.slice/crio-46906117093d39e83452dbb9633af4940d1e6be2e2775969c06bd882ca0050d5 WatchSource:0}: Error finding container 46906117093d39e83452dbb9633af4940d1e6be2e2775969c06bd882ca0050d5: Status 404 returned error can't find the container with id 46906117093d39e83452dbb9633af4940d1e6be2e2775969c06bd882ca0050d5
Jul 01 19:21:32.226971 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:32.226935    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2f63e26_b149_4c9e_b712_69e1add49e6c.slice/crio-1ac9f4b35a2ed60f498958cb531cc8e6a93156b9de300975c4ed8b7cc1ad5ae3 WatchSource:0}: Error finding container 1ac9f4b35a2ed60f498958cb531cc8e6a93156b9de300975c4ed8b7cc1ad5ae3: Status 404 returned error can't find the container with id 1ac9f4b35a2ed60f498958cb531cc8e6a93156b9de300975c4ed8b7cc1ad5ae3
Jul 01 19:21:32.228323 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:32.228285    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a8e8c5_fc6c_49cf_8726_3b0cf31d8464.slice/crio-6a5a19ad51b4941b383904a22fc6c055d9ebae61c397769c16cc1d7db8049f7f WatchSource:0}: Error finding container 6a5a19ad51b4941b383904a22fc6c055d9ebae61c397769c16cc1d7db8049f7f: Status 404 returned error can't find the container with id 6a5a19ad51b4941b383904a22fc6c055d9ebae61c397769c16cc1d7db8049f7f
Jul 01 19:21:32.230346 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:32.230317    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda709ebfa_f726_4206_acdc_2e39ba823a35.slice/crio-121ff512dde3134b350a3c95ca9bf22c408b6dc9b7ba5bd9f5575de56847223c WatchSource:0}: Error finding container 121ff512dde3134b350a3c95ca9bf22c408b6dc9b7ba5bd9f5575de56847223c: Status 404 returned error can't find the container with id 121ff512dde3134b350a3c95ca9bf22c408b6dc9b7ba5bd9f5575de56847223c
Jul 01 19:21:32.231140 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:32.231121    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00551914_29ac_4574_a3f3_b5e78dd6dca4.slice/crio-4b24b0e92021987df4f8bbfb96609b6efff4061d4b22b77e04fbf902e3344fb7 WatchSource:0}: Error finding container 4b24b0e92021987df4f8bbfb96609b6efff4061d4b22b77e04fbf902e3344fb7: Status 404 returned error can't find the container with id 4b24b0e92021987df4f8bbfb96609b6efff4061d4b22b77e04fbf902e3344fb7
Jul 01 19:21:32.233368 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:32.233324    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod617fc997_a13b_4f86_96ea_378a55762874.slice/crio-ccd806ca3e08ef4dfbbfc41f9dd9ddecd8a0c7b094ba1efed434e14cccdf0ac5 WatchSource:0}: Error finding container ccd806ca3e08ef4dfbbfc41f9dd9ddecd8a0c7b094ba1efed434e14cccdf0ac5: Status 404 returned error can't find the container with id ccd806ca3e08ef4dfbbfc41f9dd9ddecd8a0c7b094ba1efed434e14cccdf0ac5
Jul 01 19:21:32.234061 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:32.234035    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d38693f_5c9b_4e27_8894_7b190acc75db.slice/crio-f4faf2f8daa48d84dea1ade004d6f8c114ca1887a95b42f1c810adb3ec611bee WatchSource:0}: Error finding container f4faf2f8daa48d84dea1ade004d6f8c114ca1887a95b42f1c810adb3ec611bee: Status 404 returned error can't find the container with id f4faf2f8daa48d84dea1ade004d6f8c114ca1887a95b42f1c810adb3ec611bee
Jul 01 19:21:32.236088 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:32.235786    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8556e73_373f_49ed_b16d_a1c51b253357.slice/crio-cc2e374f91c2a4d77377871dd4a1b201bfdb35ee2f81ec9bd249a086597e9800 WatchSource:0}: Error finding container cc2e374f91c2a4d77377871dd4a1b201bfdb35ee2f81ec9bd249a086597e9800: Status 404 returned error can't find the container with id cc2e374f91c2a4d77377871dd4a1b201bfdb35ee2f81ec9bd249a086597e9800
Jul 01 19:21:32.237355 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:21:32.237292    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22da25a_46e6_4260_886b_98c2a65d9bbf.slice/crio-f5ef5167f808471c9e676bfe10d77b9392db63f78dccdde23567f911340c4059 WatchSource:0}: Error finding container f5ef5167f808471c9e676bfe10d77b9392db63f78dccdde23567f911340c4059: Status 404 returned error can't find the container with id f5ef5167f808471c9e676bfe10d77b9392db63f78dccdde23567f911340c4059
Jul 01 19:21:32.765362 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:32.765068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvqrk\" (UniqueName: \"kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk\") pod \"network-check-target-595cz\" (UID: \"e93ed59e-2406-4e87-a299-26c95b66a0d2\") " pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:32.766024 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:32.765499    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:21:32.766024 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:32.765519    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:21:32.766024 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:32.765531    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kvqrk for pod openshift-network-diagnostics/network-check-target-595cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:21:32.766024 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:32.765606    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk podName:e93ed59e-2406-4e87-a299-26c95b66a0d2 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:34.765570783 +0000 UTC m=+6.000696086 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvqrk" (UniqueName: "kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk") pod "network-check-target-595cz" (UID: "e93ed59e-2406-4e87-a299-26c95b66a0d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:21:32.867389 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:32.866602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:32.867389 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:32.866667    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:32.867389 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:32.866799    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:21:32.867389 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:32.866861    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret podName:85fa991c-3b94-4f96-ab9b-608b29ad2b20 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:34.866843795 +0000 UTC m=+6.101969086 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret") pod "global-pull-secret-syncer-n9vnj" (UID: "85fa991c-3b94-4f96-ab9b-608b29ad2b20") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:21:32.867389 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:32.867296    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:21:32.867389 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:32.867350    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs podName:161462f4-ad95-4e5a-ab54-148c05190926 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:34.867335514 +0000 UTC m=+6.102460823 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs") pod "network-metrics-daemon-lw425" (UID: "161462f4-ad95-4e5a-ab54-148c05190926") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:21:33.210856 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.210745    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:33.211030 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.210744    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:33.211030 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.210899    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:33.211030 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:33.210898    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:33.211271 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:33.211159    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:33.211271 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:33.211239    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:33.225412 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.225363    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 19:16:31 +0000 UTC" deadline="2027-12-30 13:06:31.90777809 +0000 UTC"
Jul 01 19:21:33.225412 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.225402    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13121h44m58.682381178s"
Jul 01 19:21:33.238651 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.238614    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nftfz" event={"ID":"b2f63e26-b149-4c9e-b712-69e1add49e6c","Type":"ContainerStarted","Data":"1ac9f4b35a2ed60f498958cb531cc8e6a93156b9de300975c4ed8b7cc1ad5ae3"}
Jul 01 19:21:33.253564 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.253468    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw6c6" event={"ID":"314e23aa-a01f-48a2-9b68-ed39d403485c","Type":"ContainerStarted","Data":"46906117093d39e83452dbb9633af4940d1e6be2e2775969c06bd882ca0050d5"}
Jul 01 19:21:33.266316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.266276    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-218.ec2.internal" event={"ID":"5bf0f4d1f6b89cb065876b678658105c","Type":"ContainerStarted","Data":"4ca82f6ed9d9e99341cb276d78976957b921b647d5ff9a05dd09cb9a18ac64c7"}
Jul 01 19:21:33.273110 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.273077    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z2d87" event={"ID":"c22da25a-46e6-4260-886b-98c2a65d9bbf","Type":"ContainerStarted","Data":"f5ef5167f808471c9e676bfe10d77b9392db63f78dccdde23567f911340c4059"}
Jul 01 19:21:33.284891 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.284829    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-218.ec2.internal" podStartSLOduration=3.284807773 podStartE2EDuration="3.284807773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:33.283907726 +0000 UTC m=+4.519033040" watchObservedRunningTime="2026-07-01 19:21:33.284807773 +0000 UTC m=+4.519933085"
Jul 01 19:21:33.285752 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.285668    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mdvx6" event={"ID":"9d38693f-5c9b-4e27-8894-7b190acc75db","Type":"ContainerStarted","Data":"f4faf2f8daa48d84dea1ade004d6f8c114ca1887a95b42f1c810adb3ec611bee"}
Jul 01 19:21:33.291373 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.291292    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2t8qd" event={"ID":"617fc997-a13b-4f86-96ea-378a55762874","Type":"ContainerStarted","Data":"ccd806ca3e08ef4dfbbfc41f9dd9ddecd8a0c7b094ba1efed434e14cccdf0ac5"}
Jul 01 19:21:33.299784 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.299683    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng" event={"ID":"a709ebfa-f726-4206-acdc-2e39ba823a35","Type":"ContainerStarted","Data":"121ff512dde3134b350a3c95ca9bf22c408b6dc9b7ba5bd9f5575de56847223c"}
Jul 01 19:21:33.313480 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.313443    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hmg26" event={"ID":"78a8e8c5-fc6c-49cf-8726-3b0cf31d8464","Type":"ContainerStarted","Data":"6a5a19ad51b4941b383904a22fc6c055d9ebae61c397769c16cc1d7db8049f7f"}
Jul 01 19:21:33.333925 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.333882    2581 generic.go:356] "Generic (PLEG): container finished" podID="d4752b2898576bdb47b1cf0f1601ae0e" containerID="0a703ad92694dc55de0452018e99717161e395c1124621465fd2ade9dd35f4f4" exitCode=0
Jul 01 19:21:33.334084 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.334050    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal" event={"ID":"d4752b2898576bdb47b1cf0f1601ae0e","Type":"ContainerDied","Data":"0a703ad92694dc55de0452018e99717161e395c1124621465fd2ade9dd35f4f4"}
Jul 01 19:21:33.345494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.345404    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rdn6x" event={"ID":"b8556e73-373f-49ed-b16d-a1c51b253357","Type":"ContainerStarted","Data":"cc2e374f91c2a4d77377871dd4a1b201bfdb35ee2f81ec9bd249a086597e9800"}
Jul 01 19:21:33.363119 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:33.362977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48" event={"ID":"00551914-29ac-4574-a3f3-b5e78dd6dca4","Type":"ContainerStarted","Data":"4b24b0e92021987df4f8bbfb96609b6efff4061d4b22b77e04fbf902e3344fb7"}
Jul 01 19:21:34.383011 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:34.382937    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal" event={"ID":"d4752b2898576bdb47b1cf0f1601ae0e","Type":"ContainerStarted","Data":"6c87836bd0306a8dc56390d9ae4016b01b6d0f67d1b068b26d0a884da0742dcf"}
Jul 01 19:21:34.786571 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:34.786482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvqrk\" (UniqueName: \"kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk\") pod \"network-check-target-595cz\" (UID: \"e93ed59e-2406-4e87-a299-26c95b66a0d2\") " pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:34.786779 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:34.786685    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:21:34.786779 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:34.786710    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:21:34.786779 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:34.786722    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kvqrk for pod openshift-network-diagnostics/network-check-target-595cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:21:34.786951 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:34.786795    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk podName:e93ed59e-2406-4e87-a299-26c95b66a0d2 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:38.786776507 +0000 UTC m=+10.021901812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvqrk" (UniqueName: "kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk") pod "network-check-target-595cz" (UID: "e93ed59e-2406-4e87-a299-26c95b66a0d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:21:34.887229 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:34.887191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:34.887396 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:34.887255    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:34.887450 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:34.887411    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:21:34.887498 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:34.887470    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret podName:85fa991c-3b94-4f96-ab9b-608b29ad2b20 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:38.887453116 +0000 UTC m=+10.122578421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret") pod "global-pull-secret-syncer-n9vnj" (UID: "85fa991c-3b94-4f96-ab9b-608b29ad2b20") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:21:34.887901 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:34.887881    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:21:34.888013 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:34.887936    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs podName:161462f4-ad95-4e5a-ab54-148c05190926 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:38.887919759 +0000 UTC m=+10.123045052 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs") pod "network-metrics-daemon-lw425" (UID: "161462f4-ad95-4e5a-ab54-148c05190926") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:21:35.210643 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:35.209877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:35.210643 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:35.210017    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:35.210643 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:35.210373    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:35.210643 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:35.210473    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:35.211269 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:35.211089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:35.211269 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:35.211186    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:37.210679 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:37.210554    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:37.211133 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:37.210794    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:37.211133 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:37.210646    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:37.211133 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:37.210949    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:37.211133 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:37.210978    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:37.211133 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:37.211039    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:38.822890 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:38.822852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvqrk\" (UniqueName: \"kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk\") pod \"network-check-target-595cz\" (UID: \"e93ed59e-2406-4e87-a299-26c95b66a0d2\") " pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:38.823340 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:38.823071    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:21:38.823340 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:38.823088    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:21:38.823340 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:38.823100    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kvqrk for pod openshift-network-diagnostics/network-check-target-595cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:21:38.823340 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:38.823162    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk podName:e93ed59e-2406-4e87-a299-26c95b66a0d2 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:46.823144344 +0000 UTC m=+18.058269659 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvqrk" (UniqueName: "kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk") pod "network-check-target-595cz" (UID: "e93ed59e-2406-4e87-a299-26c95b66a0d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:21:38.923966 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:38.923925    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:38.924141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:38.923995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:38.924200 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:38.924138    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:21:38.924200 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:38.924155    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:21:38.924305 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:38.924205    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret podName:85fa991c-3b94-4f96-ab9b-608b29ad2b20 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:46.924187568 +0000 UTC m=+18.159312857 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret") pod "global-pull-secret-syncer-n9vnj" (UID: "85fa991c-3b94-4f96-ab9b-608b29ad2b20") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:21:38.924305 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:38.924224    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs podName:161462f4-ad95-4e5a-ab54-148c05190926 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:46.924214865 +0000 UTC m=+18.159340157 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs") pod "network-metrics-daemon-lw425" (UID: "161462f4-ad95-4e5a-ab54-148c05190926") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:21:39.211853 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:39.211059    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:39.211853 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:39.211172    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:39.211853 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:39.211531    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:39.211853 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:39.211647    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:39.211853 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:39.211692    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:39.211853 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:39.211759    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:41.210812 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:41.210766    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:41.210812 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:41.210814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:41.211393 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:41.210766    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:41.211393 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:41.210919    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:41.211393 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:41.210999    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:41.211393 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:41.211073    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:43.210345 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:43.210314    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:43.210726 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:43.210354    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:43.210726 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:43.210447    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:43.210726 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:43.210556    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:43.210726 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:43.210612    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:43.210726 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:43.210695    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:45.209556 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:45.209491    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:45.209556 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:45.209518    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:45.210093 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:45.209649    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:45.210093 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:45.209748    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:45.210093 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:45.209784    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:45.210093 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:45.209865    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:46.883086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:46.883045    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvqrk\" (UniqueName: \"kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk\") pod \"network-check-target-595cz\" (UID: \"e93ed59e-2406-4e87-a299-26c95b66a0d2\") " pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:46.883636 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:46.883243    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:21:46.883636 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:46.883275    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:21:46.883636 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:46.883290    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kvqrk for pod openshift-network-diagnostics/network-check-target-595cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:21:46.883636 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:46.883351    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk podName:e93ed59e-2406-4e87-a299-26c95b66a0d2 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:02.883332717 +0000 UTC m=+34.118458020 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvqrk" (UniqueName: "kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk") pod "network-check-target-595cz" (UID: "e93ed59e-2406-4e87-a299-26c95b66a0d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:21:46.984174 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:46.984137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:46.984174 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:46.984180    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:46.984359 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:46.984280    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:21:46.984359 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:46.984282    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:21:46.984359 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:46.984334    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret podName:85fa991c-3b94-4f96-ab9b-608b29ad2b20 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:02.984318697 +0000 UTC m=+34.219443987 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret") pod "global-pull-secret-syncer-n9vnj" (UID: "85fa991c-3b94-4f96-ab9b-608b29ad2b20") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:21:46.984359 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:46.984349    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs podName:161462f4-ad95-4e5a-ab54-148c05190926 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:02.984343354 +0000 UTC m=+34.219468643 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs") pod "network-metrics-daemon-lw425" (UID: "161462f4-ad95-4e5a-ab54-148c05190926") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:21:47.210268 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:47.210183    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:47.210431 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:47.210183    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:47.210431 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:47.210313    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:47.210431 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:47.210183    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:47.210623 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:47.210414    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:47.210623 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:47.210531    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:49.210227 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:49.210189    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:49.210781 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:49.210277    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:49.210781 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:49.210375    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:49.210781 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:49.210495    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:49.210781 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:49.210539    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:49.210781 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:49.210652    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:51.211513 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.210806    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:51.211513 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:51.210946    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:51.211513 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.211357    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:51.211513 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:51.211461    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:51.212357 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.212205    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:51.212357 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:51.212319    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:51.416751 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.416717    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48" event={"ID":"00551914-29ac-4574-a3f3-b5e78dd6dca4","Type":"ContainerStarted","Data":"73f34db677e2ceb2d5aa2efb099755e2f3d91e4208a028a6445e80bf4a479f96"}
Jul 01 19:21:51.418369 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.418339    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nftfz" event={"ID":"b2f63e26-b149-4c9e-b712-69e1add49e6c","Type":"ContainerStarted","Data":"0d72c64b4f2aad3e50eb53765dd29f5f8bb129b34bb3dca7345252392dd30df7"}
Jul 01 19:21:51.420030 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.420004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw6c6" event={"ID":"314e23aa-a01f-48a2-9b68-ed39d403485c","Type":"ContainerStarted","Data":"e8491c25248708831be9596c764542374f4931d01752eff74bf0ca1f7724003d"}
Jul 01 19:21:51.421379 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.421357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z2d87" event={"ID":"c22da25a-46e6-4260-886b-98c2a65d9bbf","Type":"ContainerStarted","Data":"a600fc7d6a5d2e682ef9d61e6629ba23da58545eb2fb0866f81654577f3a9090"}
Jul 01 19:21:51.422741 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.422718    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2t8qd" event={"ID":"617fc997-a13b-4f86-96ea-378a55762874","Type":"ContainerStarted","Data":"3a7fc62c4831f8e03a61160dacdbb23d9a1677239a2c1f9609e95f6dbc9d7fc8"}
Jul 01 19:21:51.424031 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.424013    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng" event={"ID":"a709ebfa-f726-4206-acdc-2e39ba823a35","Type":"ContainerStarted","Data":"95fbbbf5e6957f04ff2d531218fa18b0d0aa3e4e15c6a48d84e9d14d64d763dc"}
Jul 01 19:21:51.425145 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.425130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hmg26" event={"ID":"78a8e8c5-fc6c-49cf-8726-3b0cf31d8464","Type":"ContainerStarted","Data":"8d23fcb4f6daa0630a9956cb49bd711c3b833d5ca77101809b72bc5bf19f013a"}
Jul 01 19:21:51.426306 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.426286    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rdn6x" event={"ID":"b8556e73-373f-49ed-b16d-a1c51b253357","Type":"ContainerStarted","Data":"18e78b27afcdd511d504895106b9ba53abcb7b9ff094b3709d4d982d74619616"}
Jul 01 19:21:51.453934 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.453769    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-218.ec2.internal" podStartSLOduration=21.453744871 podStartE2EDuration="21.453744871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:34.403006396 +0000 UTC m=+5.638131708" watchObservedRunningTime="2026-07-01 19:21:51.453744871 +0000 UTC m=+22.688870184"
Jul 01 19:21:51.454066 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.453995    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nftfz" podStartSLOduration=3.440748584 podStartE2EDuration="22.453980225s" totalImagesPullingTime="19.013231641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:51.451868022 +0000 UTC m=+22.686993345" watchObservedRunningTime="2026-07-01 19:21:51.453980225 +0000 UTC m=+22.689105538"
Jul 01 19:21:51.472074 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.472012    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-z2d87" podStartSLOduration=3.566593274 podStartE2EDuration="22.471989554s" totalImagesPullingTime="18.90539628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:51.471629728 +0000 UTC m=+22.706755060" watchObservedRunningTime="2026-07-01 19:21:51.471989554 +0000 UTC m=+22.707114869"
Jul 01 19:21:51.486739 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.486695    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rdn6x" podStartSLOduration=3.554350694 podStartE2EDuration="22.486677421s" totalImagesPullingTime="18.932326727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:51.486275715 +0000 UTC m=+22.721401028" watchObservedRunningTime="2026-07-01 19:21:51.486677421 +0000 UTC m=+22.721802733"
Jul 01 19:21:51.501103 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.501058    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hmg26" podStartSLOduration=3.553417656 podStartE2EDuration="22.501041687s" totalImagesPullingTime="18.947624031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:51.500806115 +0000 UTC m=+22.735931428" watchObservedRunningTime="2026-07-01 19:21:51.501041687 +0000 UTC m=+22.736166998"
Jul 01 19:21:51.552847 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:51.552783    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2t8qd" podStartSLOduration=3.6036528519999997 podStartE2EDuration="22.552760412s" totalImagesPullingTime="18.94910756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:51.552183898 +0000 UTC m=+22.787309211" watchObservedRunningTime="2026-07-01 19:21:51.552760412 +0000 UTC m=+22.787885725"
Jul 01 19:21:52.430444 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:52.430265    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:21:52.431265 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:52.430743    2581 generic.go:356] "Generic (PLEG): container finished" podID="00551914-29ac-4574-a3f3-b5e78dd6dca4" containerID="840cdc6b848ad72bec0a57ec18113960a06a251963b800d1c67acf4a48fadde7" exitCode=1
Jul 01 19:21:52.431265 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:52.430790    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48" event={"ID":"00551914-29ac-4574-a3f3-b5e78dd6dca4","Type":"ContainerStarted","Data":"8ee011b121e32c61e90ac58b04b929ffea7c90fb4fc6fc10d6424661a45ea4c7"}
Jul 01 19:21:52.431265 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:52.430894    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48" event={"ID":"00551914-29ac-4574-a3f3-b5e78dd6dca4","Type":"ContainerStarted","Data":"b8dc6ccb715b0983023c9a6540837b49d6b621b41910916db7b7994caaf4a878"}
Jul 01 19:21:52.431265 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:52.430929    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48" event={"ID":"00551914-29ac-4574-a3f3-b5e78dd6dca4","Type":"ContainerStarted","Data":"202e6e39b16f3ae12f6b24ec9a253dfce9f6b73f1ef86e938efba4088267c6c4"}
Jul 01 19:21:52.431265 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:52.430970    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48" event={"ID":"00551914-29ac-4574-a3f3-b5e78dd6dca4","Type":"ContainerStarted","Data":"a74c99053d1317c22702dbf3598d2b04b79084433203531c926984efc00b5164"}
Jul 01 19:21:52.431265 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:52.431009    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48" event={"ID":"00551914-29ac-4574-a3f3-b5e78dd6dca4","Type":"ContainerDied","Data":"840cdc6b848ad72bec0a57ec18113960a06a251963b800d1c67acf4a48fadde7"}
Jul 01 19:21:52.432078 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:52.432055    2581 generic.go:356] "Generic (PLEG): container finished" podID="314e23aa-a01f-48a2-9b68-ed39d403485c" containerID="e8491c25248708831be9596c764542374f4931d01752eff74bf0ca1f7724003d" exitCode=0
Jul 01 19:21:52.432183 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:52.432121    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw6c6" event={"ID":"314e23aa-a01f-48a2-9b68-ed39d403485c","Type":"ContainerDied","Data":"e8491c25248708831be9596c764542374f4931d01752eff74bf0ca1f7724003d"}
Jul 01 19:21:52.433558 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:52.433504    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mdvx6" event={"ID":"9d38693f-5c9b-4e27-8894-7b190acc75db","Type":"ContainerStarted","Data":"4747217c01b748e9f1fac2c3bc1a07b86b0b9f1f39af7917b6a39ec34ae0b79a"}
Jul 01 19:21:52.474912 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:52.474869    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mdvx6" podStartSLOduration=4.537796445 podStartE2EDuration="23.474854541s" totalImagesPullingTime="18.937058096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:52.474734695 +0000 UTC m=+23.709860019" watchObservedRunningTime="2026-07-01 19:21:52.474854541 +0000 UTC m=+23.709979853"
Jul 01 19:21:52.879187 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:52.879157    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 01 19:21:53.202089 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:53.201931    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T19:21:52.879178225Z","UUID":"8f7f6d5d-e001-4638-b86e-1a977bf6cf65","Handler":null,"Name":"","Endpoint":""}
Jul 01 19:21:53.203945 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:53.203918    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 01 19:21:53.204094 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:53.203960    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 01 19:21:53.210172 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:53.210148    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:53.210284 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:53.210148    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:53.210284 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:53.210279    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:53.210396 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:53.210283    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:53.210467 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:53.210392    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:53.210520 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:53.210483    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:53.439491 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:53.439450    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng" event={"ID":"a709ebfa-f726-4206-acdc-2e39ba823a35","Type":"ContainerStarted","Data":"93438a271cd54859cee7625e146a181c438792c91d9aa23c63faceea32e91ec6"}
Jul 01 19:21:54.444011 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:54.443783    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng" event={"ID":"a709ebfa-f726-4206-acdc-2e39ba823a35","Type":"ContainerStarted","Data":"a56df5f1b5154b4ca0f9aea262598285a69146aab63b22fa73fd953ca0238dfc"}
Jul 01 19:21:54.447017 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:54.446985    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:21:54.447413 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:54.447390    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48" event={"ID":"00551914-29ac-4574-a3f3-b5e78dd6dca4","Type":"ContainerStarted","Data":"ffee142c444d1aa97232733311842207d38103305a5f6c9e9a1162953fcc8cb7"}
Jul 01 19:21:54.466680 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:54.466633    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbvng" podStartSLOduration=4.146320216 podStartE2EDuration="25.466614323s" totalImagesPullingTime="21.320294107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:54.466099684 +0000 UTC m=+25.701225020" watchObservedRunningTime="2026-07-01 19:21:54.466614323 +0000 UTC m=+25.701739637"
Jul 01 19:21:55.210329 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:55.210254    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:55.210569 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:55.210263    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:55.210569 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:55.210388    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:55.210569 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:55.210384    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:55.210569 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:55.210509    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:55.210801 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:55.210594    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:55.846939 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:55.846896    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-z2d87"
Jul 01 19:21:55.847646 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:55.847511    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-z2d87"
Jul 01 19:21:56.452442 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:56.452412    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-z2d87"
Jul 01 19:21:56.452988 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:56.452971    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z2d87"
Jul 01 19:21:57.209744 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:57.209564    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:57.210440 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:57.209564    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:57.210440 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:57.209909    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:57.210440 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:57.209805    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:57.210440 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:57.209566    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:57.210440 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:57.209994    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:57.457942 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:57.457875    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:21:57.458275 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:57.458255    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48" event={"ID":"00551914-29ac-4574-a3f3-b5e78dd6dca4","Type":"ContainerStarted","Data":"cccc0b51e669e97ccf0b3eeca0434a5cbaed6649f630fff88fb17c6cbd75bc83"}
Jul 01 19:21:57.458751 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:57.458619    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:57.458751 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:57.458691    2581 scope.go:117] "RemoveContainer" containerID="840cdc6b848ad72bec0a57ec18113960a06a251963b800d1c67acf4a48fadde7"
Jul 01 19:21:57.458751 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:57.458699    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:57.460171 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:57.460148    2581 generic.go:356] "Generic (PLEG): container finished" podID="314e23aa-a01f-48a2-9b68-ed39d403485c" containerID="85bc99adea38c3fca101a826199ae956b3117428731f883a373e1b7253f2766d" exitCode=0
Jul 01 19:21:57.460267 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:57.460232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw6c6" event={"ID":"314e23aa-a01f-48a2-9b68-ed39d403485c","Type":"ContainerDied","Data":"85bc99adea38c3fca101a826199ae956b3117428731f883a373e1b7253f2766d"}
Jul 01 19:21:57.474868 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:57.474850    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:58.465883 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:58.465857    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:21:58.466434 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:58.466174    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48" event={"ID":"00551914-29ac-4574-a3f3-b5e78dd6dca4","Type":"ContainerStarted","Data":"0b4434df29620a93ad5c195b3e967bc549d6e6c61db79445b0aef5e390b7207f"}
Jul 01 19:21:58.466434 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:58.466360    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:58.481822 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:58.481787    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:21:58.549214 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:58.549149    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48" podStartSLOduration=10.530153004 podStartE2EDuration="29.549129355s" totalImagesPullingTime="19.018976351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:58.501713123 +0000 UTC m=+29.736838452" watchObservedRunningTime="2026-07-01 19:21:58.549129355 +0000 UTC m=+29.784254668"
Jul 01 19:21:58.852058 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:58.851968    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n9vnj"]
Jul 01 19:21:58.852210 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:58.852156    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:21:58.852285 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:58.852251    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:21:58.853649 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:58.853595    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-595cz"]
Jul 01 19:21:58.853768 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:58.853695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:21:58.853820 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:58.853774    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:21:58.868910 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:58.868887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lw425"]
Jul 01 19:21:58.869035 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:58.869005    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:21:58.869129 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:21:58.869102    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:21:59.470852 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:59.470820    2581 generic.go:356] "Generic (PLEG): container finished" podID="314e23aa-a01f-48a2-9b68-ed39d403485c" containerID="94be50a1c5ada7cdd0ff3ab78b98c37cdf516ab2099ec4791bc2af7eab2e7925" exitCode=0
Jul 01 19:21:59.471252 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:21:59.470900    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw6c6" event={"ID":"314e23aa-a01f-48a2-9b68-ed39d403485c","Type":"ContainerDied","Data":"94be50a1c5ada7cdd0ff3ab78b98c37cdf516ab2099ec4791bc2af7eab2e7925"}
Jul 01 19:22:00.210197 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:00.210166    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:22:00.210197 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:00.210192    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:22:00.210426 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:00.210317    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:22:00.210487 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:00.210442    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:22:01.210053 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:01.209896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:22:01.210540 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:01.210129    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:22:01.477702 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:01.477623    2581 generic.go:356] "Generic (PLEG): container finished" podID="314e23aa-a01f-48a2-9b68-ed39d403485c" containerID="325d2c7bd76cc34125ea75f79805210d80bed6f96750787ee77debdfff1d9f42" exitCode=0
Jul 01 19:22:01.477702 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:01.477677    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw6c6" event={"ID":"314e23aa-a01f-48a2-9b68-ed39d403485c","Type":"ContainerDied","Data":"325d2c7bd76cc34125ea75f79805210d80bed6f96750787ee77debdfff1d9f42"}
Jul 01 19:22:02.210361 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:02.210324    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:22:02.210951 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:02.210472    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n9vnj" podUID="85fa991c-3b94-4f96-ab9b-608b29ad2b20"
Jul 01 19:22:02.210951 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:02.210528    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:22:02.210951 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:02.210676    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:22:02.898266 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:02.898235    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvqrk\" (UniqueName: \"kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk\") pod \"network-check-target-595cz\" (UID: \"e93ed59e-2406-4e87-a299-26c95b66a0d2\") " pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:22:02.898433 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:02.898414    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 19:22:02.898506 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:02.898440    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 19:22:02.898506 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:02.898453    2581 projected.go:196] Error preparing data for projected volume kube-api-access-kvqrk for pod openshift-network-diagnostics/network-check-target-595cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:22:02.898640 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:02.898519    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk podName:e93ed59e-2406-4e87-a299-26c95b66a0d2 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:34.898500218 +0000 UTC m=+66.133625531 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvqrk" (UniqueName: "kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk") pod "network-check-target-595cz" (UID: "e93ed59e-2406-4e87-a299-26c95b66a0d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 19:22:02.998741 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:02.998709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:22:02.998931 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:02.998761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:22:02.998931 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:02.998864    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:22:02.998931 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:02.998895    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 19:22:02.999057 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:02.998934    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs podName:161462f4-ad95-4e5a-ab54-148c05190926 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:34.998912117 +0000 UTC m=+66.234037410 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs") pod "network-metrics-daemon-lw425" (UID: "161462f4-ad95-4e5a-ab54-148c05190926") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 19:22:02.999057 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:02.998953    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret podName:85fa991c-3b94-4f96-ab9b-608b29ad2b20 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:34.998943851 +0000 UTC m=+66.234069144 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret") pod "global-pull-secret-syncer-n9vnj" (UID: "85fa991c-3b94-4f96-ab9b-608b29ad2b20") : object "kube-system"/"original-pull-secret" not registered
Jul 01 19:22:03.209923 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:03.209886    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:22:03.210087 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:03.210002    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-595cz" podUID="e93ed59e-2406-4e87-a299-26c95b66a0d2"
Jul 01 19:22:04.012968 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.012884    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-218.ec2.internal" event="NodeReady"
Jul 01 19:22:04.013462 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.013053    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 01 19:22:04.051281 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.051253    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"]
Jul 01 19:22:04.054054 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.054035    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:22:04.058445 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.058408    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-b6cpl\""
Jul 01 19:22:04.058719 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.058701    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 01 19:22:04.059126 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.059102    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 01 19:22:04.059343 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.059323    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67b8f5c454-d8tkg"]
Jul 01 19:22:04.061467 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.061444    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-2d6jr"]
Jul 01 19:22:04.061624 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.061601    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.067279 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.063633    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-2d6jr"
Jul 01 19:22:04.067780 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.067732    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 01 19:22:04.067942 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.067921    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 01 19:22:04.068065 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.067961    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 01 19:22:04.068065 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.068009    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p4r4m\""
Jul 01 19:22:04.069111 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.069082    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 01 19:22:04.069209 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.069123    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-tgvs6\""
Jul 01 19:22:04.069209 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.069090    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 01 19:22:04.073348 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.073323    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 01 19:22:04.077277 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.077255    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"]
Jul 01 19:22:04.077453 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.077440    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fj7pv"]
Jul 01 19:22:04.080719 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.080264    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zvhnr"]
Jul 01 19:22:04.080719 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.080446    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fj7pv"
Jul 01 19:22:04.082375 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.082350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67b8f5c454-d8tkg"]
Jul 01 19:22:04.082485 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.082475    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.083789 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.083675    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rk24x\""
Jul 01 19:22:04.083921 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.083900    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 01 19:22:04.084049 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.084027    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 01 19:22:04.084152 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.084067    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 01 19:22:04.085025 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.085007    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 01 19:22:04.085216 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.085197    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2g8hv\""
Jul 01 19:22:04.085370 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.085356    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 01 19:22:04.090558 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.090534    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fj7pv"]
Jul 01 19:22:04.095505 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.095483    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zvhnr"]
Jul 01 19:22:04.096671 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.096650    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-2d6jr"]
Jul 01 19:22:04.208498 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208465    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/759856dd-e062-4e68-bbb5-9e17011f717e-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:22:04.208498 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208502    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-tmp-dir\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.208740 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0-cert\") pod \"ingress-canary-fj7pv\" (UID: \"3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0\") " pod="openshift-ingress-canary/ingress-canary-fj7pv"
Jul 01 19:22:04.208740 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-ca-trust-extracted\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.208740 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208637    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xssx7\" (UniqueName: \"kubernetes.io/projected/3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0-kube-api-access-xssx7\") pod \"ingress-canary-fj7pv\" (UID: \"3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0\") " pod="openshift-ingress-canary/ingress-canary-fj7pv"
Jul 01 19:22:04.208740 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208660    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv9z8\" (UniqueName: \"kubernetes.io/projected/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-kube-api-access-sv9z8\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.208740 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208714    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-config-volume\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.208740 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208737    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-image-registry-private-configuration\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.208995 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208757    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.208995 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208772    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.208995 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208786    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:22:04.208995 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-trusted-ca\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.208995 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208823    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg5h5\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-kube-api-access-dg5h5\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.208995 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208885    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-installation-pull-secrets\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.208995 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-bound-sa-token\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.208995 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.208988    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjslp\" (UniqueName: \"kubernetes.io/projected/c1047257-8b95-4fcc-8c65-c9101d8250c0-kube-api-access-zjslp\") pod \"network-check-source-56499946bc-2d6jr\" (UID: \"c1047257-8b95-4fcc-8c65-c9101d8250c0\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-2d6jr"
Jul 01 19:22:04.209396 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.209014    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-certificates\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.209472 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.209453    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:22:04.209636 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.209619    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:22:04.212902 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.212883    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 19:22:04.213025 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.212883    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 19:22:04.213927 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.213903    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tjdnw\""
Jul 01 19:22:04.310132 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.310132 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.310132 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310114    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:22:04.310414 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-trusted-ca\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.310414 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg5h5\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-kube-api-access-dg5h5\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.310414 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-installation-pull-secrets\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.310414 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-bound-sa-token\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.310414 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.310239    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:22:04.310414 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.310252    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:22:04.310414 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.310275    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b8f5c454-d8tkg: secret "image-registry-tls" not found
Jul 01 19:22:04.310414 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.310313    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls podName:c9d9a753-e701-4ae6-9c8a-41217d9a8f23 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:04.810293507 +0000 UTC m=+36.045418813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls") pod "dns-default-zvhnr" (UID: "c9d9a753-e701-4ae6-9c8a-41217d9a8f23") : secret "dns-default-metrics-tls" not found
Jul 01 19:22:04.310414 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.310340    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls podName:219b505f-ef8f-48a4-b1d8-ced51e82b7ba nodeName:}" failed. No retries permitted until 2026-07-01 19:22:04.810321502 +0000 UTC m=+36.045446809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls") pod "image-registry-67b8f5c454-d8tkg" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba") : secret "image-registry-tls" not found
Jul 01 19:22:04.310414 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.310349    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 19:22:04.310414 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.310413    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert podName:759856dd-e062-4e68-bbb5-9e17011f717e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:04.810386768 +0000 UTC m=+36.045512061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-wc78x" (UID: "759856dd-e062-4e68-bbb5-9e17011f717e") : secret "networking-console-plugin-cert" not found
Jul 01 19:22:04.311007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjslp\" (UniqueName: \"kubernetes.io/projected/c1047257-8b95-4fcc-8c65-c9101d8250c0-kube-api-access-zjslp\") pod \"network-check-source-56499946bc-2d6jr\" (UID: \"c1047257-8b95-4fcc-8c65-c9101d8250c0\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-2d6jr"
Jul 01 19:22:04.311007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-certificates\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.311007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/759856dd-e062-4e68-bbb5-9e17011f717e-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:22:04.311007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310565    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-tmp-dir\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.311007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0-cert\") pod \"ingress-canary-fj7pv\" (UID: \"3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0\") " pod="openshift-ingress-canary/ingress-canary-fj7pv"
Jul 01 19:22:04.311007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-ca-trust-extracted\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.311007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xssx7\" (UniqueName: \"kubernetes.io/projected/3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0-kube-api-access-xssx7\") pod \"ingress-canary-fj7pv\" (UID: \"3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0\") " pod="openshift-ingress-canary/ingress-canary-fj7pv"
Jul 01 19:22:04.311007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310708    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv9z8\" (UniqueName: \"kubernetes.io/projected/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-kube-api-access-sv9z8\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.311007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310815    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-config-volume\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.311007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.310846    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-image-registry-private-configuration\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.311459 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.311281    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-tmp-dir\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.311459 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.311375    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/759856dd-e062-4e68-bbb5-9e17011f717e-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:22:04.311569 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.311526    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-trusted-ca\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.311652 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.311612    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-ca-trust-extracted\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.311889 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.311869    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-config-volume\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.311986 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.311891    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-certificates\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.315093 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.315070    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0-cert\") pod \"ingress-canary-fj7pv\" (UID: \"3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0\") " pod="openshift-ingress-canary/ingress-canary-fj7pv"
Jul 01 19:22:04.315182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.315098    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-image-registry-private-configuration\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.315182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.315075    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-installation-pull-secrets\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.320842 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.320791    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjslp\" (UniqueName: \"kubernetes.io/projected/c1047257-8b95-4fcc-8c65-c9101d8250c0-kube-api-access-zjslp\") pod \"network-check-source-56499946bc-2d6jr\" (UID: \"c1047257-8b95-4fcc-8c65-c9101d8250c0\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-2d6jr"
Jul 01 19:22:04.320842 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.320797    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xssx7\" (UniqueName: \"kubernetes.io/projected/3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0-kube-api-access-xssx7\") pod \"ingress-canary-fj7pv\" (UID: \"3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0\") " pod="openshift-ingress-canary/ingress-canary-fj7pv"
Jul 01 19:22:04.321373 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.321349    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg5h5\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-kube-api-access-dg5h5\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.321803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.321760    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv9z8\" (UniqueName: \"kubernetes.io/projected/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-kube-api-access-sv9z8\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.322816 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.322783    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-bound-sa-token\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.383870 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.383827    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-2d6jr"
Jul 01 19:22:04.395409 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.395222    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fj7pv"
Jul 01 19:22:04.815691 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.815655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:04.815691 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.815696    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:04.815941 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:04.815725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:22:04.815941 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.815844    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:22:04.815941 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.815866    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 19:22:04.815941 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.815848    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:22:04.815941 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.815926    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b8f5c454-d8tkg: secret "image-registry-tls" not found
Jul 01 19:22:04.815941 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.815931    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls podName:c9d9a753-e701-4ae6-9c8a-41217d9a8f23 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:05.815907856 +0000 UTC m=+37.051033149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls") pod "dns-default-zvhnr" (UID: "c9d9a753-e701-4ae6-9c8a-41217d9a8f23") : secret "dns-default-metrics-tls" not found
Jul 01 19:22:04.816208 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.815952    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls podName:219b505f-ef8f-48a4-b1d8-ced51e82b7ba nodeName:}" failed. No retries permitted until 2026-07-01 19:22:05.815940331 +0000 UTC m=+37.051065621 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls") pod "image-registry-67b8f5c454-d8tkg" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba") : secret "image-registry-tls" not found
Jul 01 19:22:04.816208 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:04.815967    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert podName:759856dd-e062-4e68-bbb5-9e17011f717e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:05.815960621 +0000 UTC m=+37.051085911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-wc78x" (UID: "759856dd-e062-4e68-bbb5-9e17011f717e") : secret "networking-console-plugin-cert" not found
Jul 01 19:22:05.210103 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:05.210065    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:22:05.213205 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:05.213178    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-27xv5\""
Jul 01 19:22:05.481890 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:05.481865    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-2d6jr"]
Jul 01 19:22:05.486376 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:22:05.485740    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1047257_8b95_4fcc_8c65_c9101d8250c0.slice/crio-92b40e97a3e51607c9a5e7cbd79a0952e49d7e7cb156b35c58d9df3d1bdf2cdf WatchSource:0}: Error finding container 92b40e97a3e51607c9a5e7cbd79a0952e49d7e7cb156b35c58d9df3d1bdf2cdf: Status 404 returned error can't find the container with id 92b40e97a3e51607c9a5e7cbd79a0952e49d7e7cb156b35c58d9df3d1bdf2cdf
Jul 01 19:22:05.486376 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:05.486234    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fj7pv"]
Jul 01 19:22:05.489238 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:22:05.489200    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc22e1b_c242_4bc2_9a66_f3c5d9b7d8b0.slice/crio-7756c8084d9dee246305c0d948d23fcb75e801b7be2dca2ac8e3be1d04345442 WatchSource:0}: Error finding container 7756c8084d9dee246305c0d948d23fcb75e801b7be2dca2ac8e3be1d04345442: Status 404 returned error can't find the container with id 7756c8084d9dee246305c0d948d23fcb75e801b7be2dca2ac8e3be1d04345442
Jul 01 19:22:05.824555 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:05.824463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:05.824555 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:05.824502    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:05.824555 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:05.824526    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:22:05.824777 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:05.824647    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:22:05.824777 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:05.824655    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:22:05.824777 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:05.824698    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 19:22:05.824777 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:05.824714    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls podName:c9d9a753-e701-4ae6-9c8a-41217d9a8f23 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:07.824699943 +0000 UTC m=+39.059825236 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls") pod "dns-default-zvhnr" (UID: "c9d9a753-e701-4ae6-9c8a-41217d9a8f23") : secret "dns-default-metrics-tls" not found
Jul 01 19:22:05.824777 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:05.824739    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert podName:759856dd-e062-4e68-bbb5-9e17011f717e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:07.824726074 +0000 UTC m=+39.059851368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-wc78x" (UID: "759856dd-e062-4e68-bbb5-9e17011f717e") : secret "networking-console-plugin-cert" not found
Jul 01 19:22:05.824777 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:05.824659    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b8f5c454-d8tkg: secret "image-registry-tls" not found
Jul 01 19:22:05.824777 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:05.824766    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls podName:219b505f-ef8f-48a4-b1d8-ced51e82b7ba nodeName:}" failed. No retries permitted until 2026-07-01 19:22:07.824757885 +0000 UTC m=+39.059883174 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls") pod "image-registry-67b8f5c454-d8tkg" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba") : secret "image-registry-tls" not found
Jul 01 19:22:06.500028 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:06.499980    2581 generic.go:356] "Generic (PLEG): container finished" podID="314e23aa-a01f-48a2-9b68-ed39d403485c" containerID="eab180543a868b0cca7c5e1800fca518e5a9925354befbc29bbce87d8cb99367" exitCode=0
Jul 01 19:22:06.500473 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:06.500332    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw6c6" event={"ID":"314e23aa-a01f-48a2-9b68-ed39d403485c","Type":"ContainerDied","Data":"eab180543a868b0cca7c5e1800fca518e5a9925354befbc29bbce87d8cb99367"}
Jul 01 19:22:06.501679 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:06.501644    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-2d6jr" event={"ID":"c1047257-8b95-4fcc-8c65-c9101d8250c0","Type":"ContainerStarted","Data":"92b40e97a3e51607c9a5e7cbd79a0952e49d7e7cb156b35c58d9df3d1bdf2cdf"}
Jul 01 19:22:06.503485 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:06.503115    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fj7pv" event={"ID":"3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0","Type":"ContainerStarted","Data":"7756c8084d9dee246305c0d948d23fcb75e801b7be2dca2ac8e3be1d04345442"}
Jul 01 19:22:07.509347 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:07.509103    2581 generic.go:356] "Generic (PLEG): container finished" podID="314e23aa-a01f-48a2-9b68-ed39d403485c" containerID="af6aa6fb6fd127911e50151aa4239fd941b4b039d4d14a7a3d29c004f0c997a1" exitCode=0
Jul 01 19:22:07.509762 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:07.509187    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw6c6" event={"ID":"314e23aa-a01f-48a2-9b68-ed39d403485c","Type":"ContainerDied","Data":"af6aa6fb6fd127911e50151aa4239fd941b4b039d4d14a7a3d29c004f0c997a1"}
Jul 01 19:22:07.845190 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:07.845103    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:07.845190 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:07.845158    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:07.845190 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:07.845189    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:22:07.845440 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:07.845300    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:22:07.845440 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:07.845325    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b8f5c454-d8tkg: secret "image-registry-tls" not found
Jul 01 19:22:07.845440 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:07.845330    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 19:22:07.845440 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:07.845358    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:22:07.845440 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:07.845397    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls podName:219b505f-ef8f-48a4-b1d8-ced51e82b7ba nodeName:}" failed. No retries permitted until 2026-07-01 19:22:11.845376051 +0000 UTC m=+43.080501342 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls") pod "image-registry-67b8f5c454-d8tkg" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba") : secret "image-registry-tls" not found
Jul 01 19:22:07.845440 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:07.845425    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert podName:759856dd-e062-4e68-bbb5-9e17011f717e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:11.84540479 +0000 UTC m=+43.080530087 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-wc78x" (UID: "759856dd-e062-4e68-bbb5-9e17011f717e") : secret "networking-console-plugin-cert" not found
Jul 01 19:22:07.845440 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:07.845444    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls podName:c9d9a753-e701-4ae6-9c8a-41217d9a8f23 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:11.845433 +0000 UTC m=+43.080558294 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls") pod "dns-default-zvhnr" (UID: "c9d9a753-e701-4ae6-9c8a-41217d9a8f23") : secret "dns-default-metrics-tls" not found
Jul 01 19:22:09.518950 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:09.518916    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw6c6" event={"ID":"314e23aa-a01f-48a2-9b68-ed39d403485c","Type":"ContainerStarted","Data":"a6990ca9e0bbee32bc162071bc145517284930a11a91100c6ab4700236634433"}
Jul 01 19:22:09.520552 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:09.520516    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-2d6jr" event={"ID":"c1047257-8b95-4fcc-8c65-c9101d8250c0","Type":"ContainerStarted","Data":"3a3418906b3246c0a5e373c820d9516c29bcdaba58ef294f1256dee0c6e4852e"}
Jul 01 19:22:09.522002 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:09.521978    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fj7pv" event={"ID":"3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0","Type":"ContainerStarted","Data":"43bec07b3a1f150310e78640a87dced152d93b53ecfd114258787b94d9d870a1"}
Jul 01 19:22:09.548958 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:09.548909    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vw6c6" podStartSLOduration=11.340904412 podStartE2EDuration="40.54889482s" totalImagesPullingTime="29.207990408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:09.547915677 +0000 UTC m=+40.783041008" watchObservedRunningTime="2026-07-01 19:22:09.54889482 +0000 UTC m=+40.784020132"
Jul 01 19:22:09.599944 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:09.599826    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fj7pv" podStartSLOduration=1.747758197 podStartE2EDuration="5.599808277s" totalImagesPullingTime="3.85205008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:09.599013182 +0000 UTC m=+40.834138496" watchObservedRunningTime="2026-07-01 19:22:09.599808277 +0000 UTC m=+40.834933589"
Jul 01 19:22:09.600269 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:09.600238    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56499946bc-2d6jr" podStartSLOduration=36.73526308 podStartE2EDuration="40.600228803s" totalImagesPullingTime="3.864965723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:09.577003732 +0000 UTC m=+40.812129069" watchObservedRunningTime="2026-07-01 19:22:09.600228803 +0000 UTC m=+40.835354115"
Jul 01 19:22:11.877984 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:11.877941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:11.877984 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:11.877984    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:11.878389 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:11.878004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:22:11.878389 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:11.878099    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:22:11.878389 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:11.878134    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 19:22:11.878389 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:11.878103    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:22:11.878389 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:11.878172    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls podName:c9d9a753-e701-4ae6-9c8a-41217d9a8f23 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:19.878157654 +0000 UTC m=+51.113282947 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls") pod "dns-default-zvhnr" (UID: "c9d9a753-e701-4ae6-9c8a-41217d9a8f23") : secret "dns-default-metrics-tls" not found
Jul 01 19:22:11.878389 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:11.878190    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert podName:759856dd-e062-4e68-bbb5-9e17011f717e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:19.878181383 +0000 UTC m=+51.113306673 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-wc78x" (UID: "759856dd-e062-4e68-bbb5-9e17011f717e") : secret "networking-console-plugin-cert" not found
Jul 01 19:22:11.878389 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:11.878174    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b8f5c454-d8tkg: secret "image-registry-tls" not found
Jul 01 19:22:11.878389 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:11.878238    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls podName:219b505f-ef8f-48a4-b1d8-ced51e82b7ba nodeName:}" failed. No retries permitted until 2026-07-01 19:22:19.878229511 +0000 UTC m=+51.113354801 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls") pod "image-registry-67b8f5c454-d8tkg" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba") : secret "image-registry-tls" not found
Jul 01 19:22:19.936201 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:19.936154    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:19.936201 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:19.936204    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:19.936778 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:19.936229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:22:19.936778 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:19.936382    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 19:22:19.936778 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:19.936390    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:22:19.936778 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:19.936483    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert podName:759856dd-e062-4e68-bbb5-9e17011f717e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:35.936450605 +0000 UTC m=+67.171575911 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-wc78x" (UID: "759856dd-e062-4e68-bbb5-9e17011f717e") : secret "networking-console-plugin-cert" not found
Jul 01 19:22:19.936778 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:19.936381    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:22:19.936778 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:19.936509    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b8f5c454-d8tkg: secret "image-registry-tls" not found
Jul 01 19:22:19.936778 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:19.936531    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls podName:c9d9a753-e701-4ae6-9c8a-41217d9a8f23 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:35.936504203 +0000 UTC m=+67.171629493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls") pod "dns-default-zvhnr" (UID: "c9d9a753-e701-4ae6-9c8a-41217d9a8f23") : secret "dns-default-metrics-tls" not found
Jul 01 19:22:19.936778 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:19.936551    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls podName:219b505f-ef8f-48a4-b1d8-ced51e82b7ba nodeName:}" failed. No retries permitted until 2026-07-01 19:22:35.936539477 +0000 UTC m=+67.171664773 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls") pod "image-registry-67b8f5c454-d8tkg" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba") : secret "image-registry-tls" not found
Jul 01 19:22:30.492441 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:30.492408    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mhs48"
Jul 01 19:22:34.953747 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:34.953708    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvqrk\" (UniqueName: \"kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk\") pod \"network-check-target-595cz\" (UID: \"e93ed59e-2406-4e87-a299-26c95b66a0d2\") " pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:22:34.957511 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:34.957490    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvqrk\" (UniqueName: \"kubernetes.io/projected/e93ed59e-2406-4e87-a299-26c95b66a0d2-kube-api-access-kvqrk\") pod \"network-check-target-595cz\" (UID: \"e93ed59e-2406-4e87-a299-26c95b66a0d2\") " pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:22:35.055003 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.054955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:22:35.055003 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.055011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:22:35.065529 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.065500    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 19:22:35.066105 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.066086    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 19:22:35.075827 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:35.075801    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 01 19:22:35.075902 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:35.075890    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs podName:161462f4-ad95-4e5a-ab54-148c05190926 nodeName:}" failed. No retries permitted until 2026-07-01 19:23:39.075866939 +0000 UTC m=+130.310992230 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs") pod "network-metrics-daemon-lw425" (UID: "161462f4-ad95-4e5a-ab54-148c05190926") : secret "metrics-daemon-secret" not found
Jul 01 19:22:35.077998 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.077978    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fa991c-3b94-4f96-ab9b-608b29ad2b20-original-pull-secret\") pod \"global-pull-secret-syncer-n9vnj\" (UID: \"85fa991c-3b94-4f96-ab9b-608b29ad2b20\") " pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:22:35.126414 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.126365    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n9vnj"
Jul 01 19:22:35.225655 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.225630    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-27xv5\""
Jul 01 19:22:35.232918 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.232889    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:22:35.255165 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.255100    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n9vnj"]
Jul 01 19:22:35.259168 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:22:35.259144    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85fa991c_3b94_4f96_ab9b_608b29ad2b20.slice/crio-93ddab58836f98d4f21b8299d74d379de56eb05a58781593bff43ca358f82a19 WatchSource:0}: Error finding container 93ddab58836f98d4f21b8299d74d379de56eb05a58781593bff43ca358f82a19: Status 404 returned error can't find the container with id 93ddab58836f98d4f21b8299d74d379de56eb05a58781593bff43ca358f82a19
Jul 01 19:22:35.369252 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.369218    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-595cz"]
Jul 01 19:22:35.372105 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:22:35.372082    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode93ed59e_2406_4e87_a299_26c95b66a0d2.slice/crio-030ca74f43fa95a53c747f765ec0e73372ab0c9b3aa3b61b1d2f18327e2be239 WatchSource:0}: Error finding container 030ca74f43fa95a53c747f765ec0e73372ab0c9b3aa3b61b1d2f18327e2be239: Status 404 returned error can't find the container with id 030ca74f43fa95a53c747f765ec0e73372ab0c9b3aa3b61b1d2f18327e2be239
Jul 01 19:22:35.582767 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.582729    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n9vnj" event={"ID":"85fa991c-3b94-4f96-ab9b-608b29ad2b20","Type":"ContainerStarted","Data":"93ddab58836f98d4f21b8299d74d379de56eb05a58781593bff43ca358f82a19"}
Jul 01 19:22:35.584191 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.584167    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-595cz" event={"ID":"e93ed59e-2406-4e87-a299-26c95b66a0d2","Type":"ContainerStarted","Data":"d01689f7e3f18cd61b18193feedf19524024c92c7fdef20ed6dc9e57880c8cf1"}
Jul 01 19:22:35.584306 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.584203    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-595cz" event={"ID":"e93ed59e-2406-4e87-a299-26c95b66a0d2","Type":"ContainerStarted","Data":"030ca74f43fa95a53c747f765ec0e73372ab0c9b3aa3b61b1d2f18327e2be239"}
Jul 01 19:22:35.584306 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.584299    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:22:35.611286 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.611240    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-595cz" podStartSLOduration=66.611225397 podStartE2EDuration="1m6.611225397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:35.610928592 +0000 UTC m=+66.846053932" watchObservedRunningTime="2026-07-01 19:22:35.611225397 +0000 UTC m=+66.846350703"
Jul 01 19:22:35.963507 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.963474    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:22:35.963956 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.963513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:22:35.963956 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:35.963541    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:22:35.963956 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:35.963642    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:22:35.963956 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:35.963651    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:22:35.963956 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:35.963713    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls podName:c9d9a753-e701-4ae6-9c8a-41217d9a8f23 nodeName:}" failed. No retries permitted until 2026-07-01 19:23:07.963695799 +0000 UTC m=+99.198821090 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls") pod "dns-default-zvhnr" (UID: "c9d9a753-e701-4ae6-9c8a-41217d9a8f23") : secret "dns-default-metrics-tls" not found
Jul 01 19:22:35.963956 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:35.963655    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 19:22:35.963956 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:35.963810    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert podName:759856dd-e062-4e68-bbb5-9e17011f717e nodeName:}" failed. No retries permitted until 2026-07-01 19:23:07.963788086 +0000 UTC m=+99.198913396 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-wc78x" (UID: "759856dd-e062-4e68-bbb5-9e17011f717e") : secret "networking-console-plugin-cert" not found
Jul 01 19:22:35.963956 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:35.963658    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b8f5c454-d8tkg: secret "image-registry-tls" not found
Jul 01 19:22:35.963956 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:22:35.963859    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls podName:219b505f-ef8f-48a4-b1d8-ced51e82b7ba nodeName:}" failed. No retries permitted until 2026-07-01 19:23:07.963847369 +0000 UTC m=+99.198972660 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls") pod "image-registry-67b8f5c454-d8tkg" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba") : secret "image-registry-tls" not found
Jul 01 19:22:40.598897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:40.598860    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n9vnj" event={"ID":"85fa991c-3b94-4f96-ab9b-608b29ad2b20","Type":"ContainerStarted","Data":"c4b361e0cc69eb95c399379e4dcaf7e25bcca90ce0190f3024165ca7b389e662"}
Jul 01 19:22:40.615088 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:22:40.615033    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n9vnj" podStartSLOduration=67.268217729 podStartE2EDuration="1m11.615018078s" totalImagesPullingTime="4.346800349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:40.614251001 +0000 UTC m=+71.849376350" watchObservedRunningTime="2026-07-01 19:22:40.615018078 +0000 UTC m=+71.850143461"
Jul 01 19:23:06.588980 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:23:06.588951    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-595cz"
Jul 01 19:23:07.995012 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:23:07.994952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:23:07.995012 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:23:07.995011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:23:07.995426 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:23:07.995032    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:23:07.995426 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:23:07.995121    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:23:07.995426 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:23:07.995150    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 19:23:07.995426 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:23:07.995175    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:23:07.995426 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:23:07.995194    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b8f5c454-d8tkg: secret "image-registry-tls" not found
Jul 01 19:23:07.995426 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:23:07.995202    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls podName:c9d9a753-e701-4ae6-9c8a-41217d9a8f23 nodeName:}" failed. No retries permitted until 2026-07-01 19:24:11.99518599 +0000 UTC m=+163.230311280 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls") pod "dns-default-zvhnr" (UID: "c9d9a753-e701-4ae6-9c8a-41217d9a8f23") : secret "dns-default-metrics-tls" not found
Jul 01 19:23:07.995426 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:23:07.995218    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert podName:759856dd-e062-4e68-bbb5-9e17011f717e nodeName:}" failed. No retries permitted until 2026-07-01 19:24:11.995210631 +0000 UTC m=+163.230335921 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-wc78x" (UID: "759856dd-e062-4e68-bbb5-9e17011f717e") : secret "networking-console-plugin-cert" not found
Jul 01 19:23:07.995426 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:23:07.995240    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls podName:219b505f-ef8f-48a4-b1d8-ced51e82b7ba nodeName:}" failed. No retries permitted until 2026-07-01 19:24:11.995226745 +0000 UTC m=+163.230352040 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls") pod "image-registry-67b8f5c454-d8tkg" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba") : secret "image-registry-tls" not found
Jul 01 19:23:39.115237 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:23:39.115198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:23:39.115751 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:23:39.115352    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 01 19:23:39.115751 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:23:39.115419    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs podName:161462f4-ad95-4e5a-ab54-148c05190926 nodeName:}" failed. No retries permitted until 2026-07-01 19:25:41.11540162 +0000 UTC m=+252.350526920 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs") pod "network-metrics-daemon-lw425" (UID: "161462f4-ad95-4e5a-ab54-148c05190926") : secret "metrics-daemon-secret" not found
Jul 01 19:24:07.066619 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:24:07.066535    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x" podUID="759856dd-e062-4e68-bbb5-9e17011f717e"
Jul 01 19:24:07.074904 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:24:07.074867    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg" podUID="219b505f-ef8f-48a4-b1d8-ced51e82b7ba"
Jul 01 19:24:07.101167 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:24:07.101138    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zvhnr" podUID="c9d9a753-e701-4ae6-9c8a-41217d9a8f23"
Jul 01 19:24:07.221338 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:24:07.221301    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:24:07.809728 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:24:07.809701    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:24:07.809892 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:24:07.809702    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:24:07.809892 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:24:07.809702    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:24:12.046538 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:24:12.046483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:24:12.046538 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:24:12.046537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:24:12.046993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:24:12.046558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:24:12.046993 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:24:12.046657    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:24:12.046993 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:24:12.046683    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b8f5c454-d8tkg: secret "image-registry-tls" not found
Jul 01 19:24:12.046993 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:24:12.046661    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 19:24:12.046993 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:24:12.046732    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls podName:219b505f-ef8f-48a4-b1d8-ced51e82b7ba nodeName:}" failed. No retries permitted until 2026-07-01 19:26:14.046716631 +0000 UTC m=+285.281841921 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls") pod "image-registry-67b8f5c454-d8tkg" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba") : secret "image-registry-tls" not found
Jul 01 19:24:12.046993 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:24:12.046658    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:24:12.046993 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:24:12.046746    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert podName:759856dd-e062-4e68-bbb5-9e17011f717e nodeName:}" failed. No retries permitted until 2026-07-01 19:26:14.046740126 +0000 UTC m=+285.281865415 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-wc78x" (UID: "759856dd-e062-4e68-bbb5-9e17011f717e") : secret "networking-console-plugin-cert" not found
Jul 01 19:24:12.046993 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:24:12.046764    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls podName:c9d9a753-e701-4ae6-9c8a-41217d9a8f23 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:14.046751363 +0000 UTC m=+285.281876655 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls") pod "dns-default-zvhnr" (UID: "c9d9a753-e701-4ae6-9c8a-41217d9a8f23") : secret "dns-default-metrics-tls" not found
Jul 01 19:24:19.210640 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:24:19.210604    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:25:41.174292 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:41.174238    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:25:41.174812 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:41.174369    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 01 19:25:41.174812 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:41.174423    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs podName:161462f4-ad95-4e5a-ab54-148c05190926 nodeName:}" failed. No retries permitted until 2026-07-01 19:27:43.174408356 +0000 UTC m=+374.409533646 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs") pod "network-metrics-daemon-lw425" (UID: "161462f4-ad95-4e5a-ab54-148c05190926") : secret "metrics-daemon-secret" not found
Jul 01 19:25:58.718853 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.718819    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"]
Jul 01 19:25:58.721422 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.721406    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:25:58.730000 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.729979    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 01 19:25:58.730914 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.730896    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"]
Jul 01 19:25:58.733305 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.733283    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 01 19:25:58.733399 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.733283    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 01 19:25:58.733399 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.733321    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kqqps\""
Jul 01 19:25:58.733544 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.733530    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:25:58.741358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.741330    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 01 19:25:58.741478 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.741378    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 01 19:25:58.741478 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.741446    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 01 19:25:58.741478 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.741448    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-w5r7g\""
Jul 01 19:25:58.741705 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.741689    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 01 19:25:58.742373 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.742354    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"]
Jul 01 19:25:58.752744 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.752724    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"]
Jul 01 19:25:58.808895 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.808864    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"]
Jul 01 19:25:58.811794 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.811773    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"
Jul 01 19:25:58.816085 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.816055    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 01 19:25:58.816194 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.816177    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 01 19:25:58.816633 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.816612    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 01 19:25:58.816769 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.816697    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-bnx6d\""
Jul 01 19:25:58.817873 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.817657    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 01 19:25:58.834653 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.834626    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5f67b55555-gmpj2"]
Jul 01 19:25:58.837238 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.837221    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"]
Jul 01 19:25:58.837347 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.837334    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:58.839958 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.839939    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5dbf8f9b4f-qjprw"]
Jul 01 19:25:58.840066 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.840050    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"
Jul 01 19:25:58.840503 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:58.840475    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"openshift-insights-serving-cert\" is forbidden: User \"system:node:ip-10-0-138-218.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-138-218.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" type="*v1.Secret"
Jul 01 19:25:58.840962 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:58.840936    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"service-ca-bundle\" is forbidden: User \"system:node:ip-10-0-138-218.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-138-218.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" type="*v1.ConfigMap"
Jul 01 19:25:58.840962 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:58.840941    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"operator-dockercfg-fg77d\" is forbidden: User \"system:node:ip-10-0-138-218.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-138-218.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fg77d\"" type="*v1.Secret"
Jul 01 19:25:58.841200 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:58.841179    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"trusted-ca-bundle\" is forbidden: User \"system:node:ip-10-0-138-218.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-138-218.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap"
Jul 01 19:25:58.841455 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:58.841438    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-138-218.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-138-218.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 01 19:25:58.842652 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.842633    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:58.843212 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.843193    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 01 19:25:58.844460 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:58.844438    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-138-218.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-138-218.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 01 19:25:58.857459 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.857442    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 01 19:25:58.857548 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.857468    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 01 19:25:58.857611 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.857559    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-29xfk\""
Jul 01 19:25:58.857766 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.857753    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 01 19:25:58.858097 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.858080    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 01 19:25:58.858142 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.858124    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 01 19:25:58.859250 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.859237    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5pgx6\""
Jul 01 19:25:58.863066 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.863050    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 01 19:25:58.863143 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.863109    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 01 19:25:58.865114 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.865097    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"]
Jul 01 19:25:58.865940 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.865924    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 01 19:25:58.866509 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.866496    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 01 19:25:58.883743 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.883717    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"]
Jul 01 19:25:58.886807 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.886786    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-gmpj2"]
Jul 01 19:25:58.906907 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.906887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5dbf8f9b4f-qjprw"]
Jul 01 19:25:58.907164 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.907138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-lprc6\" (UID: \"e1e07fa2-5b89-4547-97f9-775a7e003a77\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:25:58.907271 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.907175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5gnc\" (UniqueName: \"kubernetes.io/projected/55638e79-3269-468e-a5c2-c8e1acd52a3d-kube-api-access-m5gnc\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q8bfb\" (UID: \"55638e79-3269-468e-a5c2-c8e1acd52a3d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"
Jul 01 19:25:58.907336 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.907266    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq6v4\" (UniqueName: \"kubernetes.io/projected/e1e07fa2-5b89-4547-97f9-775a7e003a77-kube-api-access-bq6v4\") pod \"cluster-samples-operator-9ddc5ff76-lprc6\" (UID: \"e1e07fa2-5b89-4547-97f9-775a7e003a77\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:25:58.907336 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.907326    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e10211ae-3648-4b0b-98ad-70bab9739e9b-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:25:58.907433 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.907392    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55638e79-3269-468e-a5c2-c8e1acd52a3d-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q8bfb\" (UID: \"55638e79-3269-468e-a5c2-c8e1acd52a3d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"
Jul 01 19:25:58.907433 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.907426    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pqzb\" (UniqueName: \"kubernetes.io/projected/e10211ae-3648-4b0b-98ad-70bab9739e9b-kube-api-access-9pqzb\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:25:58.907520 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.907456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55638e79-3269-468e-a5c2-c8e1acd52a3d-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q8bfb\" (UID: \"55638e79-3269-468e-a5c2-c8e1acd52a3d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"
Jul 01 19:25:58.907520 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:58.907481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:25:59.008872 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.008787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55638e79-3269-468e-a5c2-c8e1acd52a3d-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q8bfb\" (UID: \"55638e79-3269-468e-a5c2-c8e1acd52a3d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"
Jul 01 19:25:59.008872 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.008832    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-stats-auth\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.008872 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.008851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhvj9\" (UniqueName: \"kubernetes.io/projected/8077fede-71c7-4815-93e1-05c14abe96fd-kube-api-access-nhvj9\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.008872 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.008867    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-snapshots\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.009227 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.008897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pqzb\" (UniqueName: \"kubernetes.io/projected/e10211ae-3648-4b0b-98ad-70bab9739e9b-kube-api-access-9pqzb\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:25:59.009227 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.008996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f011f0f-0e35-4018-8fb0-08180a66a798-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-crtfr\" (UID: \"5f011f0f-0e35-4018-8fb0-08180a66a798\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"
Jul 01 19:25:59.009227 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9xh5\" (UniqueName: \"kubernetes.io/projected/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-kube-api-access-c9xh5\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.009227 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-service-ca-bundle\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.009227 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:25:59.009227 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009126    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-serving-cert\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.009227 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009151    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5f011f0f-0e35-4018-8fb0-08180a66a798-config\") pod \"service-ca-operator-5d7b7dc95c-crtfr\" (UID: \"5f011f0f-0e35-4018-8fb0-08180a66a798\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"
Jul 01 19:25:59.009227 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-lprc6\" (UID: \"e1e07fa2-5b89-4547-97f9-775a7e003a77\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:25:59.009227 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.009193    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 19:25:59.009227 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009213    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55638e79-3269-468e-a5c2-c8e1acd52a3d-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q8bfb\" (UID: \"55638e79-3269-468e-a5c2-c8e1acd52a3d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"
Jul 01 19:25:59.009613 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.009249    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls podName:e10211ae-3648-4b0b-98ad-70bab9739e9b nodeName:}" failed. No retries permitted until 2026-07-01 19:25:59.509234264 +0000 UTC m=+270.744359567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-c4nbt" (UID: "e10211ae-3648-4b0b-98ad-70bab9739e9b") : secret "cluster-monitoring-operator-tls" not found
Jul 01 19:25:59.009613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009273    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e10211ae-3648-4b0b-98ad-70bab9739e9b-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:25:59.009613 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.009274    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 01 19:25:59.009613 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.009353    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls podName:e1e07fa2-5b89-4547-97f9-775a7e003a77 nodeName:}" failed. No retries permitted until 2026-07-01 19:25:59.509344076 +0000 UTC m=+270.744469371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-lprc6" (UID: "e1e07fa2-5b89-4547-97f9-775a7e003a77") : secret "samples-operator-tls" not found
Jul 01 19:25:59.009613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009306    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-default-certificate\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.009613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frmss\" (UniqueName: \"kubernetes.io/projected/5f011f0f-0e35-4018-8fb0-08180a66a798-kube-api-access-frmss\") pod \"service-ca-operator-5d7b7dc95c-crtfr\" (UID: \"5f011f0f-0e35-4018-8fb0-08180a66a798\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"
Jul 01 19:25:59.009613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009408    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.009613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.009613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009444    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.009613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009476    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5gnc\" (UniqueName: \"kubernetes.io/projected/55638e79-3269-468e-a5c2-c8e1acd52a3d-kube-api-access-m5gnc\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q8bfb\" (UID: \"55638e79-3269-468e-a5c2-c8e1acd52a3d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"
Jul 01 19:25:59.009613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009511    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq6v4\" (UniqueName: \"kubernetes.io/projected/e1e07fa2-5b89-4547-97f9-775a7e003a77-kube-api-access-bq6v4\") pod \"cluster-samples-operator-9ddc5ff76-lprc6\" (UID: \"e1e07fa2-5b89-4547-97f9-775a7e003a77\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:25:59.009613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009531    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-tmp\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.010030 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009850    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55638e79-3269-468e-a5c2-c8e1acd52a3d-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q8bfb\" (UID: \"55638e79-3269-468e-a5c2-c8e1acd52a3d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"
Jul 01 19:25:59.010030 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.009966    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e10211ae-3648-4b0b-98ad-70bab9739e9b-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:25:59.011182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.011159    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55638e79-3269-468e-a5c2-c8e1acd52a3d-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q8bfb\" (UID: \"55638e79-3269-468e-a5c2-c8e1acd52a3d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"
Jul 01 19:25:59.020844 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.020820    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq6v4\" (UniqueName: \"kubernetes.io/projected/e1e07fa2-5b89-4547-97f9-775a7e003a77-kube-api-access-bq6v4\") pod \"cluster-samples-operator-9ddc5ff76-lprc6\" (UID: \"e1e07fa2-5b89-4547-97f9-775a7e003a77\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:25:59.020953 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.020883    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5gnc\" (UniqueName: \"kubernetes.io/projected/55638e79-3269-468e-a5c2-c8e1acd52a3d-kube-api-access-m5gnc\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q8bfb\" (UID: \"55638e79-3269-468e-a5c2-c8e1acd52a3d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"
Jul 01 19:25:59.020953 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.020887    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pqzb\" (UniqueName: \"kubernetes.io/projected/e10211ae-3648-4b0b-98ad-70bab9739e9b-kube-api-access-9pqzb\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:25:59.110589 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110553    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-stats-auth\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.110768 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhvj9\" (UniqueName: \"kubernetes.io/projected/8077fede-71c7-4815-93e1-05c14abe96fd-kube-api-access-nhvj9\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.110768 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110638    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-snapshots\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.110768 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110658    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f011f0f-0e35-4018-8fb0-08180a66a798-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-crtfr\" (UID: \"5f011f0f-0e35-4018-8fb0-08180a66a798\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"
Jul 01 19:25:59.110768 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9xh5\" (UniqueName: \"kubernetes.io/projected/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-kube-api-access-c9xh5\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.110768 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-service-ca-bundle\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.110768 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-serving-cert\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.110768 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5f011f0f-0e35-4018-8fb0-08180a66a798-config\") pod \"service-ca-operator-5d7b7dc95c-crtfr\" (UID: \"5f011f0f-0e35-4018-8fb0-08180a66a798\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"
Jul 01 19:25:59.111122 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-default-certificate\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.111122 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110855    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frmss\" (UniqueName: \"kubernetes.io/projected/5f011f0f-0e35-4018-8fb0-08180a66a798-kube-api-access-frmss\") pod \"service-ca-operator-5d7b7dc95c-crtfr\" (UID: \"5f011f0f-0e35-4018-8fb0-08180a66a798\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"
Jul 01 19:25:59.111122 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110885    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.111122 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.111122 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.110933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.111122 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.111026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-tmp\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.111407 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.111136    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle podName:8077fede-71c7-4815-93e1-05c14abe96fd nodeName:}" failed. No retries permitted until 2026-07-01 19:25:59.611112032 +0000 UTC m=+270.846237339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle") pod "router-default-5dbf8f9b4f-qjprw" (UID: "8077fede-71c7-4815-93e1-05c14abe96fd") : configmap references non-existent config key: service-ca.crt
Jul 01 19:25:59.111407 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.111190    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 19:25:59.111407 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.111245    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs podName:8077fede-71c7-4815-93e1-05c14abe96fd nodeName:}" failed. No retries permitted until 2026-07-01 19:25:59.611228633 +0000 UTC m=+270.846353932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs") pod "router-default-5dbf8f9b4f-qjprw" (UID: "8077fede-71c7-4815-93e1-05c14abe96fd") : secret "router-metrics-certs-default" not found
Jul 01 19:25:59.111407 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.111357    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-tmp\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.111565 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.111415    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5f011f0f-0e35-4018-8fb0-08180a66a798-config\") pod \"service-ca-operator-5d7b7dc95c-crtfr\" (UID: \"5f011f0f-0e35-4018-8fb0-08180a66a798\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"
Jul 01 19:25:59.111565 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.111430    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-snapshots\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.113213 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.113185    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f011f0f-0e35-4018-8fb0-08180a66a798-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-crtfr\" (UID: \"5f011f0f-0e35-4018-8fb0-08180a66a798\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"
Jul 01 19:25:59.113603 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.113564    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-stats-auth\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.113688 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.113630    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-default-certificate\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.121085 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.121058    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"
Jul 01 19:25:59.121272 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.121256    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frmss\" (UniqueName: \"kubernetes.io/projected/5f011f0f-0e35-4018-8fb0-08180a66a798-kube-api-access-frmss\") pod \"service-ca-operator-5d7b7dc95c-crtfr\" (UID: \"5f011f0f-0e35-4018-8fb0-08180a66a798\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"
Jul 01 19:25:59.121501 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.121484    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhvj9\" (UniqueName: \"kubernetes.io/projected/8077fede-71c7-4815-93e1-05c14abe96fd-kube-api-access-nhvj9\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.152756 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.152732    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"
Jul 01 19:25:59.251593 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.251548    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb"]
Jul 01 19:25:59.255059 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:25:59.255025    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55638e79_3269_468e_a5c2_c8e1acd52a3d.slice/crio-71d0d8a09e9df2965f133e1d85f152d2ff1a4a54977f85cffb15aaebf99c8898 WatchSource:0}: Error finding container 71d0d8a09e9df2965f133e1d85f152d2ff1a4a54977f85cffb15aaebf99c8898: Status 404 returned error can't find the container with id 71d0d8a09e9df2965f133e1d85f152d2ff1a4a54977f85cffb15aaebf99c8898
Jul 01 19:25:59.296029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.296001    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr"]
Jul 01 19:25:59.298530 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:25:59.298507    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f011f0f_0e35_4018_8fb0_08180a66a798.slice/crio-c0f4d92bdd8b7630f257b99867f952ceefd22b7fcd7272fe68b9ee44d8ca1011 WatchSource:0}: Error finding container c0f4d92bdd8b7630f257b99867f952ceefd22b7fcd7272fe68b9ee44d8ca1011: Status 404 returned error can't find the container with id c0f4d92bdd8b7630f257b99867f952ceefd22b7fcd7272fe68b9ee44d8ca1011
Jul 01 19:25:59.514970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.514889    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:25:59.514970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.514932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-lprc6\" (UID: \"e1e07fa2-5b89-4547-97f9-775a7e003a77\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:25:59.515138 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.515035    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 01 19:25:59.515138 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.515037    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 19:25:59.515138 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.515088    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls podName:e1e07fa2-5b89-4547-97f9-775a7e003a77 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:00.51507387 +0000 UTC m=+271.750199160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-lprc6" (UID: "e1e07fa2-5b89-4547-97f9-775a7e003a77") : secret "samples-operator-tls" not found
Jul 01 19:25:59.515138 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.515101    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls podName:e10211ae-3648-4b0b-98ad-70bab9739e9b nodeName:}" failed. No retries permitted until 2026-07-01 19:26:00.515094827 +0000 UTC m=+271.750220116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-c4nbt" (UID: "e10211ae-3648-4b0b-98ad-70bab9739e9b") : secret "cluster-monitoring-operator-tls" not found
Jul 01 19:25:59.615957 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.615917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.615957 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.615963    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:25:59.616156 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.616079    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle podName:8077fede-71c7-4815-93e1-05c14abe96fd nodeName:}" failed. No retries permitted until 2026-07-01 19:26:00.616062618 +0000 UTC m=+271.851187909 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle") pod "router-default-5dbf8f9b4f-qjprw" (UID: "8077fede-71c7-4815-93e1-05c14abe96fd") : configmap references non-existent config key: service-ca.crt
Jul 01 19:25:59.616156 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.616133    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 19:25:59.616228 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:25:59.616181    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs podName:8077fede-71c7-4815-93e1-05c14abe96fd nodeName:}" failed. No retries permitted until 2026-07-01 19:26:00.616168534 +0000 UTC m=+271.851293824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs") pod "router-default-5dbf8f9b4f-qjprw" (UID: "8077fede-71c7-4815-93e1-05c14abe96fd") : secret "router-metrics-certs-default" not found
Jul 01 19:25:59.688829 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.688799    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 01 19:25:59.691252 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.691233    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-service-ca-bundle\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.805134 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.805051    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 01 19:25:59.813030 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.812998    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-serving-cert\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:25:59.918207 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.918123    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 01 19:25:59.923023 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:25:59.922993    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:26:00.049213 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:00.049183    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 01 19:26:00.081323 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:00.081227    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr" event={"ID":"5f011f0f-0e35-4018-8fb0-08180a66a798","Type":"ContainerStarted","Data":"c0f4d92bdd8b7630f257b99867f952ceefd22b7fcd7272fe68b9ee44d8ca1011"}
Jul 01 19:26:00.082419 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:00.082394    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb" event={"ID":"55638e79-3269-468e-a5c2-c8e1acd52a3d","Type":"ContainerStarted","Data":"71d0d8a09e9df2965f133e1d85f152d2ff1a4a54977f85cffb15aaebf99c8898"}
Jul 01 19:26:00.138022 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:00.137992    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 01 19:26:00.146878 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:00.146847    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9xh5\" (UniqueName: \"kubernetes.io/projected/5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40-kube-api-access-c9xh5\") pod \"insights-operator-5f67b55555-gmpj2\" (UID: \"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40\") " pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:26:00.321775 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:00.321745    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fg77d\""
Jul 01 19:26:00.347161 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:00.347083    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-gmpj2"
Jul 01 19:26:00.511407 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:00.511374    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-gmpj2"]
Jul 01 19:26:00.514732 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:26:00.514701    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b6f4c0c_c227_4df9_b0b4_26cd3ae87b40.slice/crio-c5d0340c159c55c766e7835048dd36845804d24226d1f7db030f46afb5329383 WatchSource:0}: Error finding container c5d0340c159c55c766e7835048dd36845804d24226d1f7db030f46afb5329383: Status 404 returned error can't find the container with id c5d0340c159c55c766e7835048dd36845804d24226d1f7db030f46afb5329383
Jul 01 19:26:00.523486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:00.523442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:26:00.523626 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:00.523494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-lprc6\" (UID: \"e1e07fa2-5b89-4547-97f9-775a7e003a77\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:26:00.523848 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:00.523827    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 01 19:26:00.523935 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:00.523896    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls podName:e1e07fa2-5b89-4547-97f9-775a7e003a77 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:02.523875705 +0000 UTC m=+273.759001001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-lprc6" (UID: "e1e07fa2-5b89-4547-97f9-775a7e003a77") : secret "samples-operator-tls" not found
Jul 01 19:26:00.524023 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:00.524005    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 19:26:00.524084 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:00.524073    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls podName:e10211ae-3648-4b0b-98ad-70bab9739e9b nodeName:}" failed. No retries permitted until 2026-07-01 19:26:02.524054629 +0000 UTC m=+273.759179926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-c4nbt" (UID: "e10211ae-3648-4b0b-98ad-70bab9739e9b") : secret "cluster-monitoring-operator-tls" not found
Jul 01 19:26:00.624166 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:00.624071    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:00.624166 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:00.624123    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:00.624384 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:00.624247    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle podName:8077fede-71c7-4815-93e1-05c14abe96fd nodeName:}" failed. No retries permitted until 2026-07-01 19:26:02.624226218 +0000 UTC m=+273.859351540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle") pod "router-default-5dbf8f9b4f-qjprw" (UID: "8077fede-71c7-4815-93e1-05c14abe96fd") : configmap references non-existent config key: service-ca.crt
Jul 01 19:26:00.624384 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:00.624251    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 19:26:00.624384 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:00.624354    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs podName:8077fede-71c7-4815-93e1-05c14abe96fd nodeName:}" failed. No retries permitted until 2026-07-01 19:26:02.624342841 +0000 UTC m=+273.859468131 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs") pod "router-default-5dbf8f9b4f-qjprw" (UID: "8077fede-71c7-4815-93e1-05c14abe96fd") : secret "router-metrics-certs-default" not found
Jul 01 19:26:01.086699 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:01.086663    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-gmpj2" event={"ID":"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40","Type":"ContainerStarted","Data":"c5d0340c159c55c766e7835048dd36845804d24226d1f7db030f46afb5329383"}
Jul 01 19:26:02.091053 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:02.091022    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr" event={"ID":"5f011f0f-0e35-4018-8fb0-08180a66a798","Type":"ContainerStarted","Data":"96e56ee62ad8b3860ce98bb238dd42bf8fa82e68e79b2b4c811b030e18038192"}
Jul 01 19:26:02.111868 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:02.111653    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr" podStartSLOduration=2.000224691 podStartE2EDuration="4.111637543s" totalImagesPullingTime="2.111412852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:02.111595291 +0000 UTC m=+273.346720626" watchObservedRunningTime="2026-07-01 19:26:02.111637543 +0000 UTC m=+273.346762855"
Jul 01 19:26:02.537109 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:02.537066    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:26:02.537297 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:02.537130    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-lprc6\" (UID: \"e1e07fa2-5b89-4547-97f9-775a7e003a77\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:26:02.537297 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:02.537225    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 19:26:02.537297 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:02.537225    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 01 19:26:02.537443 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:02.537299    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls podName:e10211ae-3648-4b0b-98ad-70bab9739e9b nodeName:}" failed. No retries permitted until 2026-07-01 19:26:06.537278157 +0000 UTC m=+277.772403467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-c4nbt" (UID: "e10211ae-3648-4b0b-98ad-70bab9739e9b") : secret "cluster-monitoring-operator-tls" not found
Jul 01 19:26:02.537443 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:02.537339    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls podName:e1e07fa2-5b89-4547-97f9-775a7e003a77 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:06.537321162 +0000 UTC m=+277.772446455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-lprc6" (UID: "e1e07fa2-5b89-4547-97f9-775a7e003a77") : secret "samples-operator-tls" not found
Jul 01 19:26:02.637809 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:02.637773    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:02.637809 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:02.637815    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:02.638064 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:02.637974    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 19:26:02.638064 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:02.637985    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle podName:8077fede-71c7-4815-93e1-05c14abe96fd nodeName:}" failed. No retries permitted until 2026-07-01 19:26:06.637960133 +0000 UTC m=+277.873085422 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle") pod "router-default-5dbf8f9b4f-qjprw" (UID: "8077fede-71c7-4815-93e1-05c14abe96fd") : configmap references non-existent config key: service-ca.crt
Jul 01 19:26:02.638175 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:02.638093    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs podName:8077fede-71c7-4815-93e1-05c14abe96fd nodeName:}" failed. No retries permitted until 2026-07-01 19:26:06.638077817 +0000 UTC m=+277.873203123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs") pod "router-default-5dbf8f9b4f-qjprw" (UID: "8077fede-71c7-4815-93e1-05c14abe96fd") : secret "router-metrics-certs-default" not found
Jul 01 19:26:03.095182 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:03.095150    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb" event={"ID":"55638e79-3269-468e-a5c2-c8e1acd52a3d","Type":"ContainerStarted","Data":"c2a9e1ee0a0332008f85e020015062372cd671dc5671addedee3289f5eb9ea70"}
Jul 01 19:26:03.121536 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:03.121488    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb" podStartSLOduration=2.302444872 podStartE2EDuration="5.121473504s" totalImagesPullingTime="2.819028632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:03.121252186 +0000 UTC m=+274.356377511" watchObservedRunningTime="2026-07-01 19:26:03.121473504 +0000 UTC m=+274.356598816"
Jul 01 19:26:04.099644 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.099606    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-gmpj2" event={"ID":"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40","Type":"ContainerStarted","Data":"692db34d94af4ee9520c621377db0ed5663cb415c75cad097853f328fb467d94"}
Jul 01 19:26:04.122490 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.122441    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5f67b55555-gmpj2" podStartSLOduration=3.496541924 podStartE2EDuration="6.122425065s" totalImagesPullingTime="2.625883141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:04.121162728 +0000 UTC m=+275.356288055" watchObservedRunningTime="2026-07-01 19:26:04.122425065 +0000 UTC m=+275.357550388"
Jul 01 19:26:04.475343 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.475312    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kq5nv"]
Jul 01 19:26:04.483060 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.483036    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.486507 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.486482    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 01 19:26:04.486641 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.486552    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nqzwd\""
Jul 01 19:26:04.486939 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.486924    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 01 19:26:04.505880 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.505853    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kq5nv"]
Jul 01 19:26:04.555950 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.555908    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-data-volume\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.556127 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.555958    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vq2l\" (UniqueName: \"kubernetes.io/projected/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-api-access-6vq2l\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.556127 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.556036    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-crio-socket\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.556285 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.556263    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.556331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.556318    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.657300 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.657262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.657497 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.657317    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.657497 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.657341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-data-volume\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.657497 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.657361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vq2l\" (UniqueName: \"kubernetes.io/projected/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-api-access-6vq2l\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.657497 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:04.657395    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 19:26:04.657497 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:04.657448    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 19:26:04.657497 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.657458    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-crio-socket\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.657497 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.657400    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-crio-socket\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.657497 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:04.657466    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm podName:c135ff2d-fb79-4eca-8c85-5e6a1105f3c2 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:05.157445441 +0000 UTC m=+276.392570735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kq5nv" (UID: "c135ff2d-fb79-4eca-8c85-5e6a1105f3c2") : configmap "kube-rbac-proxy" not found
Jul 01 19:26:04.657801 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:04.657521    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls podName:c135ff2d-fb79-4eca-8c85-5e6a1105f3c2 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:05.157497554 +0000 UTC m=+276.392622856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kq5nv" (UID: "c135ff2d-fb79-4eca-8c85-5e6a1105f3c2") : secret "insights-runtime-extractor-tls" not found
Jul 01 19:26:04.657801 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.657675    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-data-volume\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:04.669621 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:04.669569    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vq2l\" (UniqueName: \"kubernetes.io/projected/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-api-access-6vq2l\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:05.161029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:05.160987    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:05.161525 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:05.161055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:05.161525 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:05.161080    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 19:26:05.161525 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:05.161160    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm podName:c135ff2d-fb79-4eca-8c85-5e6a1105f3c2 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:06.161137727 +0000 UTC m=+277.396263030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kq5nv" (UID: "c135ff2d-fb79-4eca-8c85-5e6a1105f3c2") : configmap "kube-rbac-proxy" not found
Jul 01 19:26:05.161525 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:05.161206    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 19:26:05.161525 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:05.161258    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls podName:c135ff2d-fb79-4eca-8c85-5e6a1105f3c2 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:06.161242144 +0000 UTC m=+277.396367440 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kq5nv" (UID: "c135ff2d-fb79-4eca-8c85-5e6a1105f3c2") : secret "insights-runtime-extractor-tls" not found
Jul 01 19:26:06.169858 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:06.169817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:06.170249 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:06.169953    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:06.170249 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:06.169976    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 19:26:06.170249 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:06.170018    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 19:26:06.170249 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:06.170042    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls podName:c135ff2d-fb79-4eca-8c85-5e6a1105f3c2 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:08.170026588 +0000 UTC m=+279.405151878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kq5nv" (UID: "c135ff2d-fb79-4eca-8c85-5e6a1105f3c2") : secret "insights-runtime-extractor-tls" not found
Jul 01 19:26:06.170249 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:06.170059    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm podName:c135ff2d-fb79-4eca-8c85-5e6a1105f3c2 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:08.170052603 +0000 UTC m=+279.405177893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kq5nv" (UID: "c135ff2d-fb79-4eca-8c85-5e6a1105f3c2") : configmap "kube-rbac-proxy" not found
Jul 01 19:26:06.572742 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:06.572689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:26:06.572929 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:06.572840    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 19:26:06.572929 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:06.572860    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-lprc6\" (UID: \"e1e07fa2-5b89-4547-97f9-775a7e003a77\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:26:06.572929 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:06.572903    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls podName:e10211ae-3648-4b0b-98ad-70bab9739e9b nodeName:}" failed. No retries permitted until 2026-07-01 19:26:14.572888584 +0000 UTC m=+285.808013877 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-c4nbt" (UID: "e10211ae-3648-4b0b-98ad-70bab9739e9b") : secret "cluster-monitoring-operator-tls" not found
Jul 01 19:26:06.573099 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:06.572953    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 01 19:26:06.573099 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:06.573023    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls podName:e1e07fa2-5b89-4547-97f9-775a7e003a77 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:14.57301548 +0000 UTC m=+285.808140770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-lprc6" (UID: "e1e07fa2-5b89-4547-97f9-775a7e003a77") : secret "samples-operator-tls" not found
Jul 01 19:26:06.673505 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:06.673463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:06.673505 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:06.673506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:06.673763 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:06.673646    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle podName:8077fede-71c7-4815-93e1-05c14abe96fd nodeName:}" failed. No retries permitted until 2026-07-01 19:26:14.673629855 +0000 UTC m=+285.908755145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle") pod "router-default-5dbf8f9b4f-qjprw" (UID: "8077fede-71c7-4815-93e1-05c14abe96fd") : configmap references non-existent config key: service-ca.crt
Jul 01 19:26:06.673763 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:06.673674    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 19:26:06.673763 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:06.673719    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs podName:8077fede-71c7-4815-93e1-05c14abe96fd nodeName:}" failed. No retries permitted until 2026-07-01 19:26:14.67370739 +0000 UTC m=+285.908832680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs") pod "router-default-5dbf8f9b4f-qjprw" (UID: "8077fede-71c7-4815-93e1-05c14abe96fd") : secret "router-metrics-certs-default" not found
Jul 01 19:26:07.266929 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:07.266904    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hmg26_78a8e8c5-fc6c-49cf-8726-3b0cf31d8464/dns-node-resolver/0.log"
Jul 01 19:26:08.186602 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:08.186548    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:08.186773 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:08.186619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:08.186773 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:08.186674    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 19:26:08.186773 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:08.186747    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm podName:c135ff2d-fb79-4eca-8c85-5e6a1105f3c2 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:12.18673162 +0000 UTC m=+283.421856910 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kq5nv" (UID: "c135ff2d-fb79-4eca-8c85-5e6a1105f3c2") : configmap "kube-rbac-proxy" not found
Jul 01 19:26:08.186773 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:08.186759    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 19:26:08.186942 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:08.186819    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls podName:c135ff2d-fb79-4eca-8c85-5e6a1105f3c2 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:12.186806518 +0000 UTC m=+283.421931812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kq5nv" (UID: "c135ff2d-fb79-4eca-8c85-5e6a1105f3c2") : secret "insights-runtime-extractor-tls" not found
Jul 01 19:26:08.467332 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:08.467305    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rdn6x_b8556e73-373f-49ed-b16d-a1c51b253357/node-ca/0.log"
Jul 01 19:26:09.067318 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:09.067239    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fj7pv_3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0/serve-healthcheck-canary/0.log"
Jul 01 19:26:09.871395 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:09.871361    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-q8bfb_55638e79-3269-468e-a5c2-c8e1acd52a3d/kube-storage-version-migrator-operator/0.log"
Jul 01 19:26:10.810432 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:10.810382    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x" podUID="759856dd-e062-4e68-bbb5-9e17011f717e"
Jul 01 19:26:10.810432 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:10.810413    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zvhnr" podUID="c9d9a753-e701-4ae6-9c8a-41217d9a8f23"
Jul 01 19:26:10.810732 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:10.810416    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg" podUID="219b505f-ef8f-48a4-b1d8-ced51e82b7ba"
Jul 01 19:26:11.120318 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:11.120237    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:26:11.120318 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:11.120266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:26:11.120746 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:11.120379    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:26:12.222461 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:12.222429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:12.222900 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:12.222472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:12.222900 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:12.222556    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 19:26:12.222900 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:12.222632    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 19:26:12.222900 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:12.222649    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm podName:c135ff2d-fb79-4eca-8c85-5e6a1105f3c2 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:20.222627065 +0000 UTC m=+291.457752356 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kq5nv" (UID: "c135ff2d-fb79-4eca-8c85-5e6a1105f3c2") : configmap "kube-rbac-proxy" not found
Jul 01 19:26:12.222900 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:12.222685    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls podName:c135ff2d-fb79-4eca-8c85-5e6a1105f3c2 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:20.222672971 +0000 UTC m=+291.457798261 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kq5nv" (UID: "c135ff2d-fb79-4eca-8c85-5e6a1105f3c2") : secret "insights-runtime-extractor-tls" not found
Jul 01 19:26:14.140411 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:14.140381    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:26:14.140795 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:14.140418    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:26:14.140795 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:14.140495    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls\") pod \"image-registry-67b8f5c454-d8tkg\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") " pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:26:14.140795 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:14.140519    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 19:26:14.140795 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:14.140599    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 19:26:14.140795 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:14.140609    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls podName:c9d9a753-e701-4ae6-9c8a-41217d9a8f23 nodeName:}" failed. No retries permitted until 2026-07-01 19:28:16.14058991 +0000 UTC m=+407.375715201 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls") pod "dns-default-zvhnr" (UID: "c9d9a753-e701-4ae6-9c8a-41217d9a8f23") : secret "dns-default-metrics-tls" not found
Jul 01 19:26:14.140795 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:14.140610    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67b8f5c454-d8tkg: secret "image-registry-tls" not found
Jul 01 19:26:14.140795 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:14.140645    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls podName:219b505f-ef8f-48a4-b1d8-ced51e82b7ba nodeName:}" failed. No retries permitted until 2026-07-01 19:28:16.140634989 +0000 UTC m=+407.375760311 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls") pod "image-registry-67b8f5c454-d8tkg" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba") : secret "image-registry-tls" not found
Jul 01 19:26:14.140795 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:14.140598    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 01 19:26:14.140795 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:14.140673    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert podName:759856dd-e062-4e68-bbb5-9e17011f717e nodeName:}" failed. No retries permitted until 2026-07-01 19:28:16.140666172 +0000 UTC m=+407.375791462 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-wc78x" (UID: "759856dd-e062-4e68-bbb5-9e17011f717e") : secret "networking-console-plugin-cert" not found
Jul 01 19:26:14.643138 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:14.643103    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:26:14.643311 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:14.643155    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-lprc6\" (UID: \"e1e07fa2-5b89-4547-97f9-775a7e003a77\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:26:14.643311 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:14.643258    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 01 19:26:14.643381 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:14.643328    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls podName:e10211ae-3648-4b0b-98ad-70bab9739e9b nodeName:}" failed. No retries permitted until 2026-07-01 19:26:30.643310018 +0000 UTC m=+301.878435316 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-c4nbt" (UID: "e10211ae-3648-4b0b-98ad-70bab9739e9b") : secret "cluster-monitoring-operator-tls" not found
Jul 01 19:26:14.645564 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:14.645532    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1e07fa2-5b89-4547-97f9-775a7e003a77-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-lprc6\" (UID: \"e1e07fa2-5b89-4547-97f9-775a7e003a77\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:26:14.744588 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:14.744550    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:14.744757 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:14.744615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:14.744757 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:14.744734    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 19:26:14.744983 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:14.744739    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle podName:8077fede-71c7-4815-93e1-05c14abe96fd nodeName:}" failed. No retries permitted until 2026-07-01 19:26:30.744720234 +0000 UTC m=+301.979845540 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle") pod "router-default-5dbf8f9b4f-qjprw" (UID: "8077fede-71c7-4815-93e1-05c14abe96fd") : configmap references non-existent config key: service-ca.crt
Jul 01 19:26:14.744983 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:14.744779    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs podName:8077fede-71c7-4815-93e1-05c14abe96fd nodeName:}" failed. No retries permitted until 2026-07-01 19:26:30.744769166 +0000 UTC m=+301.979894456 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs") pod "router-default-5dbf8f9b4f-qjprw" (UID: "8077fede-71c7-4815-93e1-05c14abe96fd") : secret "router-metrics-certs-default" not found
Jul 01 19:26:14.942245 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:14.942164    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"
Jul 01 19:26:15.069808 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:15.069786    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6"]
Jul 01 19:26:15.136755 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:15.136725    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6" event={"ID":"e1e07fa2-5b89-4547-97f9-775a7e003a77","Type":"ContainerStarted","Data":"815c5abdf503e80569ddc4c38e53786cebe479065e7021267e355157225ad5d8"}
Jul 01 19:26:18.148234 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:18.148199    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6" event={"ID":"e1e07fa2-5b89-4547-97f9-775a7e003a77","Type":"ContainerStarted","Data":"d372d0082a3d95f337fa832ffd3ea914f61bb014ac4fb75794083f5d67d6877e"}
Jul 01 19:26:18.148627 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:18.148257    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6" event={"ID":"e1e07fa2-5b89-4547-97f9-775a7e003a77","Type":"ContainerStarted","Data":"b30b823e9d45650ee8a7bc527ab6ecebefc43e022b822a5e5684c94ab40adb9f"}
Jul 01 19:26:18.168404 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:18.168353    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-lprc6" podStartSLOduration=17.59051108 podStartE2EDuration="20.168336808s" totalImagesPullingTime="2.577825728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:18.166296332 +0000 UTC m=+289.401421662" watchObservedRunningTime="2026-07-01 19:26:18.168336808 +0000 UTC m=+289.403462121"
Jul 01 19:26:20.291646 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:20.291569    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:20.291646 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:20.291662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:20.292290 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:20.291808    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 19:26:20.292290 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:20.291875    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls podName:c135ff2d-fb79-4eca-8c85-5e6a1105f3c2 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:36.291860087 +0000 UTC m=+307.526985378 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kq5nv" (UID: "c135ff2d-fb79-4eca-8c85-5e6a1105f3c2") : secret "insights-runtime-extractor-tls" not found
Jul 01 19:26:20.292290 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:20.292181    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:22.211948 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:22.211895    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-lw425" podUID="161462f4-ad95-4e5a-ab54-148c05190926"
Jul 01 19:26:29.140614 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:29.140565    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:26:29.146481 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:29.146446    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:26:29.151634 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:29.151611    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 01 19:26:30.681484 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:30.681445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:26:30.721557 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:30.683873    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e10211ae-3648-4b0b-98ad-70bab9739e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-c4nbt\" (UID: \"e10211ae-3648-4b0b-98ad-70bab9739e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:26:30.782713 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:30.782672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:30.782713 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:30.782713    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:30.783298 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:30.783277    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8077fede-71c7-4815-93e1-05c14abe96fd-service-ca-bundle\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:30.785158 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:30.785140    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8077fede-71c7-4815-93e1-05c14abe96fd-metrics-certs\") pod \"router-default-5dbf8f9b4f-qjprw\" (UID: \"8077fede-71c7-4815-93e1-05c14abe96fd\") " pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:30.833552 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:30.833519    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kqqps\""
Jul 01 19:26:30.839897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:30.839877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"
Jul 01 19:26:30.961054 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:30.961028    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5pgx6\""
Jul 01 19:26:30.968690 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:30.968651    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:30.985376 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:30.985348    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt"]
Jul 01 19:26:30.988835 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:26:30.988808    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10211ae_3648_4b0b_98ad_70bab9739e9b.slice/crio-c2d626ef70699a8ad72deff69b2d2dbdbcfa10b307f52c426e061c5cbc98b47c WatchSource:0}: Error finding container c2d626ef70699a8ad72deff69b2d2dbdbcfa10b307f52c426e061c5cbc98b47c: Status 404 returned error can't find the container with id c2d626ef70699a8ad72deff69b2d2dbdbcfa10b307f52c426e061c5cbc98b47c
Jul 01 19:26:30.991226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:30.991208    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 19:26:31.119070 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:31.119039    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5dbf8f9b4f-qjprw"]
Jul 01 19:26:31.119291 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:26:31.119269    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8077fede_71c7_4815_93e1_05c14abe96fd.slice/crio-e81ae6be5f4ffd669e8f18b01063cc3cc25b8db49990870c34f3dc858e9c1e27 WatchSource:0}: Error finding container e81ae6be5f4ffd669e8f18b01063cc3cc25b8db49990870c34f3dc858e9c1e27: Status 404 returned error can't find the container with id e81ae6be5f4ffd669e8f18b01063cc3cc25b8db49990870c34f3dc858e9c1e27
Jul 01 19:26:31.186998 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:31.186928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw" event={"ID":"8077fede-71c7-4815-93e1-05c14abe96fd","Type":"ContainerStarted","Data":"e81ae6be5f4ffd669e8f18b01063cc3cc25b8db49990870c34f3dc858e9c1e27"}
Jul 01 19:26:31.187983 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:31.187958    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt" event={"ID":"e10211ae-3648-4b0b-98ad-70bab9739e9b","Type":"ContainerStarted","Data":"c2d626ef70699a8ad72deff69b2d2dbdbcfa10b307f52c426e061c5cbc98b47c"}
Jul 01 19:26:32.192202 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:32.192163    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw" event={"ID":"8077fede-71c7-4815-93e1-05c14abe96fd","Type":"ContainerStarted","Data":"394f14ed77933c25d59356255cb3ac6909a2e9f66fd22707671cf74187c8cf9a"}
Jul 01 19:26:32.224149 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:32.224096    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw" podStartSLOduration=34.224079705 podStartE2EDuration="34.224079705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:25:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:32.223606528 +0000 UTC m=+303.458731843" watchObservedRunningTime="2026-07-01 19:26:32.224079705 +0000 UTC m=+303.459205017"
Jul 01 19:26:32.969792 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:32.969761    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:32.972765 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:32.972732    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:33.194971 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:33.194934    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:33.196316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:33.196295    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5dbf8f9b4f-qjprw"
Jul 01 19:26:34.199800 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:34.199703    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt" event={"ID":"e10211ae-3648-4b0b-98ad-70bab9739e9b","Type":"ContainerStarted","Data":"919ccc60f7efe63c31f94e82caba4aa6fac57832d5f2ea0863cd3379de7185b5"}
Jul 01 19:26:34.209734 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:34.209711    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:26:34.220009 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:34.219966    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-c4nbt" podStartSLOduration=33.40812554 podStartE2EDuration="36.219951483s" totalImagesPullingTime="2.811825943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:34.219813618 +0000 UTC m=+305.454938931" watchObservedRunningTime="2026-07-01 19:26:34.219951483 +0000 UTC m=+305.455076794"
Jul 01 19:26:34.304818 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:34.304777    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j"]
Jul 01 19:26:34.307160 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:34.307139    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j"
Jul 01 19:26:34.310146 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:34.310130    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 01 19:26:34.310248 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:34.310171    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-q4dzf\""
Jul 01 19:26:34.319042 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:34.319021    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j"]
Jul 01 19:26:34.412304 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:34.412266    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4df0155a-3ee6-4047-8a88-7d0da8422382-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-tj44j\" (UID: \"4df0155a-3ee6-4047-8a88-7d0da8422382\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j"
Jul 01 19:26:34.513397 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:34.513360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4df0155a-3ee6-4047-8a88-7d0da8422382-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-tj44j\" (UID: \"4df0155a-3ee6-4047-8a88-7d0da8422382\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j"
Jul 01 19:26:34.513619 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:34.513532    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 01 19:26:34.513697 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:34.513633    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4df0155a-3ee6-4047-8a88-7d0da8422382-tls-certificates podName:4df0155a-3ee6-4047-8a88-7d0da8422382 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:35.013611716 +0000 UTC m=+306.248737009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/4df0155a-3ee6-4047-8a88-7d0da8422382-tls-certificates") pod "prometheus-operator-admission-webhook-7ddc8784d4-tj44j" (UID: "4df0155a-3ee6-4047-8a88-7d0da8422382") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 01 19:26:35.016033 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:35.015998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4df0155a-3ee6-4047-8a88-7d0da8422382-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-tj44j\" (UID: \"4df0155a-3ee6-4047-8a88-7d0da8422382\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j"
Jul 01 19:26:35.018366 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:35.018342    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4df0155a-3ee6-4047-8a88-7d0da8422382-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-tj44j\" (UID: \"4df0155a-3ee6-4047-8a88-7d0da8422382\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j"
Jul 01 19:26:35.216589 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:35.216559    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j"
Jul 01 19:26:35.338125 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:26:35.338081    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df0155a_3ee6_4047_8a88_7d0da8422382.slice/crio-6ea0b97e4674e7f16511694a26ce81d063b0bc0f4b07b7b8f4bfd0cccb1593de WatchSource:0}: Error finding container 6ea0b97e4674e7f16511694a26ce81d063b0bc0f4b07b7b8f4bfd0cccb1593de: Status 404 returned error can't find the container with id 6ea0b97e4674e7f16511694a26ce81d063b0bc0f4b07b7b8f4bfd0cccb1593de
Jul 01 19:26:35.338125 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:35.338106    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j"]
Jul 01 19:26:36.211008 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:36.210968    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j" event={"ID":"4df0155a-3ee6-4047-8a88-7d0da8422382","Type":"ContainerStarted","Data":"6ea0b97e4674e7f16511694a26ce81d063b0bc0f4b07b7b8f4bfd0cccb1593de"}
Jul 01 19:26:36.326229 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:36.326190    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:36.328510 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:36.328478    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c135ff2d-fb79-4eca-8c85-5e6a1105f3c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kq5nv\" (UID: \"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2\") " pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:36.594659 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:36.594564    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nqzwd\""
Jul 01 19:26:36.602239 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:36.602219    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kq5nv"
Jul 01 19:26:36.761135 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:36.761091    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kq5nv"]
Jul 01 19:26:36.763490 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:26:36.763462    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc135ff2d_fb79_4eca_8c85_5e6a1105f3c2.slice/crio-6bff4e2966ea1c94fd78027cddbc86ee588fcbf93010bc83d6fa4543cf7b4d2c WatchSource:0}: Error finding container 6bff4e2966ea1c94fd78027cddbc86ee588fcbf93010bc83d6fa4543cf7b4d2c: Status 404 returned error can't find the container with id 6bff4e2966ea1c94fd78027cddbc86ee588fcbf93010bc83d6fa4543cf7b4d2c
Jul 01 19:26:37.215216 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:37.215183    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kq5nv" event={"ID":"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2","Type":"ContainerStarted","Data":"6bff4e2966ea1c94fd78027cddbc86ee588fcbf93010bc83d6fa4543cf7b4d2c"}
Jul 01 19:26:38.221145 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.221104    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j" event={"ID":"4df0155a-3ee6-4047-8a88-7d0da8422382","Type":"ContainerStarted","Data":"17bf9b5c31235343daf462cc19c20ad2e84d73229b6d2d8d987a70c4817e485f"}
Jul 01 19:26:38.221618 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.221332    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j"
Jul 01 19:26:38.226874 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.226846    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j"
Jul 01 19:26:38.253595 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.253529    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tj44j" podStartSLOduration=2.414982442 podStartE2EDuration="4.25350852s" totalImagesPullingTime="1.838526078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:38.23734598 +0000 UTC m=+309.472471312" watchObservedRunningTime="2026-07-01 19:26:38.25350852 +0000 UTC m=+309.488633842"
Jul 01 19:26:38.368625 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.368571    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"]
Jul 01 19:26:38.370884 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.370863    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:38.373976 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.373948    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 01 19:26:38.374110 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.373973    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-5w955\""
Jul 01 19:26:38.374110 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.374070    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 01 19:26:38.374110 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.374104    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 01 19:26:38.381056 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.381033    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"]
Jul 01 19:26:38.444533 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.444502    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d778352c-e062-42df-9a41-d0e8b084113d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:38.444708 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.444563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d778352c-e062-42df-9a41-d0e8b084113d-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:38.444708 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.444614    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rnfx\" (UniqueName: \"kubernetes.io/projected/d778352c-e062-42df-9a41-d0e8b084113d-kube-api-access-9rnfx\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:38.444708 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.444659    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d778352c-e062-42df-9a41-d0e8b084113d-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:38.545988 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.545907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d778352c-e062-42df-9a41-d0e8b084113d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:38.545988 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.545976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d778352c-e062-42df-9a41-d0e8b084113d-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:38.546201 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.545999    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rnfx\" (UniqueName: \"kubernetes.io/projected/d778352c-e062-42df-9a41-d0e8b084113d-kube-api-access-9rnfx\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:38.546201 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.546057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d778352c-e062-42df-9a41-d0e8b084113d-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:38.546201 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:38.546185    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 01 19:26:38.546362 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:38.546253    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d778352c-e062-42df-9a41-d0e8b084113d-prometheus-operator-tls podName:d778352c-e062-42df-9a41-d0e8b084113d nodeName:}" failed. No retries permitted until 2026-07-01 19:26:39.04623573 +0000 UTC m=+310.281361021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/d778352c-e062-42df-9a41-d0e8b084113d-prometheus-operator-tls") pod "prometheus-operator-7b7d896497-vfzq2" (UID: "d778352c-e062-42df-9a41-d0e8b084113d") : secret "prometheus-operator-tls" not found
Jul 01 19:26:38.546853 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.546805    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d778352c-e062-42df-9a41-d0e8b084113d-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:38.548866 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.548844    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d778352c-e062-42df-9a41-d0e8b084113d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:38.555670 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:38.555648    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rnfx\" (UniqueName: \"kubernetes.io/projected/d778352c-e062-42df-9a41-d0e8b084113d-kube-api-access-9rnfx\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:39.052049 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:39.052009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d778352c-e062-42df-9a41-d0e8b084113d-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:39.055082 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:39.055048    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d778352c-e062-42df-9a41-d0e8b084113d-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-vfzq2\" (UID: \"d778352c-e062-42df-9a41-d0e8b084113d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:39.282024 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:39.281981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"
Jul 01 19:26:40.728454 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:40.728431    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-vfzq2"]
Jul 01 19:26:40.730566 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:26:40.730540    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd778352c_e062_42df_9a41_d0e8b084113d.slice/crio-e71b2fad1fa561c304ae5652152ca8007ffed2917f511e6d88a4477b60494c39 WatchSource:0}: Error finding container e71b2fad1fa561c304ae5652152ca8007ffed2917f511e6d88a4477b60494c39: Status 404 returned error can't find the container with id e71b2fad1fa561c304ae5652152ca8007ffed2917f511e6d88a4477b60494c39
Jul 01 19:26:41.232028 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:41.231989    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kq5nv" event={"ID":"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2","Type":"ContainerStarted","Data":"eba2bd6486ababd59a49b5cf9d84143995c5a32808a22e15248296316047d12b"}
Jul 01 19:26:41.232967 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:41.232944    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2" event={"ID":"d778352c-e062-42df-9a41-d0e8b084113d","Type":"ContainerStarted","Data":"e71b2fad1fa561c304ae5652152ca8007ffed2917f511e6d88a4477b60494c39"}
Jul 01 19:26:43.245275 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:43.245234    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kq5nv" event={"ID":"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2","Type":"ContainerStarted","Data":"100ed522110da4b33aa89af27bf84e5b920845dadeb3362f1e6af20a2c6f882f"}
Jul 01 19:26:43.247195 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:43.247165    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2" event={"ID":"d778352c-e062-42df-9a41-d0e8b084113d","Type":"ContainerStarted","Data":"da33126e7af2ccf415e5b4343f48d2ffaa8d9239f5f125e84afaf7dfcb0e7f50"}
Jul 01 19:26:43.247328 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:43.247214    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2" event={"ID":"d778352c-e062-42df-9a41-d0e8b084113d","Type":"ContainerStarted","Data":"a9ff206f97687d0a24f6af824a3430a661a439bcbe3335d898efce02a5626671"}
Jul 01 19:26:43.268464 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:43.268411    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-vfzq2" podStartSLOduration=3.6476410379999997 podStartE2EDuration="5.26839462s" totalImagesPullingTime="1.620753582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:43.266814767 +0000 UTC m=+314.501940099" watchObservedRunningTime="2026-07-01 19:26:43.26839462 +0000 UTC m=+314.503519932"
Jul 01 19:26:45.013221 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.013191    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-x74m2"]
Jul 01 19:26:45.015568 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.015551    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.018466 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.018447    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-r7ct4\""
Jul 01 19:26:45.018673 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.018657    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 01 19:26:45.019093 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.019074    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 01 19:26:45.019318 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.019299    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 01 19:26:45.110720 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.110688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq99k\" (UniqueName: \"kubernetes.io/projected/8476b125-8a53-4baa-b109-cb06bf728010-kube-api-access-tq99k\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.110877 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.110740    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8476b125-8a53-4baa-b109-cb06bf728010-root\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.110877 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.110802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-accelerators-collector-config\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.110951 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.110881    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-wtmp\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.110951 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.110910    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.110951 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.110930    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8476b125-8a53-4baa-b109-cb06bf728010-sys\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.111035 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.110957    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-textfile\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.111068 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.111042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8476b125-8a53-4baa-b109-cb06bf728010-metrics-client-ca\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.111100 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.111066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-tls\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.211601 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.211552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.211782 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.211655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8476b125-8a53-4baa-b109-cb06bf728010-sys\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.211782 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.211697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-textfile\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.211902 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.211771    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8476b125-8a53-4baa-b109-cb06bf728010-sys\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.211902 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.211795    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8476b125-8a53-4baa-b109-cb06bf728010-metrics-client-ca\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.211902 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.211836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-tls\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.211902 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.211870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq99k\" (UniqueName: \"kubernetes.io/projected/8476b125-8a53-4baa-b109-cb06bf728010-kube-api-access-tq99k\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.212098 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.211934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8476b125-8a53-4baa-b109-cb06bf728010-root\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.212098 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.211965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-accelerators-collector-config\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.212098 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.212011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-wtmp\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.212098 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.212047    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8476b125-8a53-4baa-b109-cb06bf728010-root\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.212275 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.212125    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-textfile\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.212275 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.212268    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-wtmp\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.212461 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.212436    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8476b125-8a53-4baa-b109-cb06bf728010-metrics-client-ca\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.212637 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.212615    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-accelerators-collector-config\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.214327 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.214302    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.214516 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.214498    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8476b125-8a53-4baa-b109-cb06bf728010-node-exporter-tls\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.231244 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.231217    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq99k\" (UniqueName: \"kubernetes.io/projected/8476b125-8a53-4baa-b109-cb06bf728010-kube-api-access-tq99k\") pod \"node-exporter-x74m2\" (UID: \"8476b125-8a53-4baa-b109-cb06bf728010\") " pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.256748 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.256710    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kq5nv" event={"ID":"c135ff2d-fb79-4eca-8c85-5e6a1105f3c2","Type":"ContainerStarted","Data":"3f1d1acd0cee2eccecc708b636853b2b5c50ff15dc91031c3762ba233dbb391f"}
Jul 01 19:26:45.285821 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.285768    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kq5nv" podStartSLOduration=33.275776578 podStartE2EDuration="41.285753507s" totalImagesPullingTime="8.009976929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:26:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:45.284709014 +0000 UTC m=+316.519834326" watchObservedRunningTime="2026-07-01 19:26:45.285753507 +0000 UTC m=+316.520878818"
Jul 01 19:26:45.324831 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.324798    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x74m2"
Jul 01 19:26:45.334453 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:26:45.334426    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8476b125_8a53_4baa_b109_cb06bf728010.slice/crio-aa91b7eca0db9ad8b49551855c9e16b9c07fac4923fa583cf97af7fe6d123a05 WatchSource:0}: Error finding container aa91b7eca0db9ad8b49551855c9e16b9c07fac4923fa583cf97af7fe6d123a05: Status 404 returned error can't find the container with id aa91b7eca0db9ad8b49551855c9e16b9c07fac4923fa583cf97af7fe6d123a05
Jul 01 19:26:45.796047 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.796017    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 19:26:45.799342 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.799318    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.804086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.803731    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 01 19:26:45.804086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.803746    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 01 19:26:45.804086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.803775    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 01 19:26:45.804086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.803815    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 01 19:26:45.804086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.803826    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 01 19:26:45.804086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.803750    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 01 19:26:45.804086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.803918    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 01 19:26:45.804086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.803934    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 01 19:26:45.804086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.803969    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 01 19:26:45.804086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.803783    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-cl297\""
Jul 01 19:26:45.828371 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.828341    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 19:26:45.918247 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918207    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.918247 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2df21efc-612d-4ab4-9723-0f12a1f83a52-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.918492 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918276    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2df21efc-612d-4ab4-9723-0f12a1f83a52-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.918492 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918353    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.918492 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918409    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.918642 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2df21efc-612d-4ab4-9723-0f12a1f83a52-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.918642 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918538    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-web-config\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.918642 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918626    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2df21efc-612d-4ab4-9723-0f12a1f83a52-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.918748 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918679    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-config-volume\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.918748 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918710    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.918815 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918743    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.918855 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918813    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2df21efc-612d-4ab4-9723-0f12a1f83a52-config-out\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:45.918855 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:45.918840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wfzw\" (UniqueName: \"kubernetes.io/projected/2df21efc-612d-4ab4-9723-0f12a1f83a52-kube-api-access-7wfzw\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.019949 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.019907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wfzw\" (UniqueName: \"kubernetes.io/projected/2df21efc-612d-4ab4-9723-0f12a1f83a52-kube-api-access-7wfzw\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.020375 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.019965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.020375 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.019997    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2df21efc-612d-4ab4-9723-0f12a1f83a52-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.020375 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.020027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2df21efc-612d-4ab4-9723-0f12a1f83a52-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.020375 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.020051    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.020375 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.020077    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.020375 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:46.020089    2581 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 01 19:26:46.020375 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:46.020166    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-main-tls podName:2df21efc-612d-4ab4-9723-0f12a1f83a52 nodeName:}" failed. No retries permitted until 2026-07-01 19:26:46.520143567 +0000 UTC m=+317.755268857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52") : secret "alertmanager-main-tls" not found
Jul 01 19:26:46.020375 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.020263    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2df21efc-612d-4ab4-9723-0f12a1f83a52-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.020769 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.020547    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-web-config\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.020769 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.020649    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2df21efc-612d-4ab4-9723-0f12a1f83a52-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.020769 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.020697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-config-volume\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.020769 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.020752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.020960 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.020786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.020960 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.020880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2df21efc-612d-4ab4-9723-0f12a1f83a52-config-out\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.021189 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.021159    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2df21efc-612d-4ab4-9723-0f12a1f83a52-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.021319 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.021222    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2df21efc-612d-4ab4-9723-0f12a1f83a52-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.022272 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.022246    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2df21efc-612d-4ab4-9723-0f12a1f83a52-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.026615 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.024086    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.029486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.029460    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2df21efc-612d-4ab4-9723-0f12a1f83a52-config-out\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.029680 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.029660    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2df21efc-612d-4ab4-9723-0f12a1f83a52-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.029907 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.029879    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.030553 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.030529    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-web-config\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.030674 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.030644    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.031071 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.031040    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.031684 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.031663    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-config-volume\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.032927 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.032906    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wfzw\" (UniqueName: \"kubernetes.io/projected/2df21efc-612d-4ab4-9723-0f12a1f83a52-kube-api-access-7wfzw\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.261717 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.261680    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x74m2" event={"ID":"8476b125-8a53-4baa-b109-cb06bf728010","Type":"ContainerStarted","Data":"aa91b7eca0db9ad8b49551855c9e16b9c07fac4923fa583cf97af7fe6d123a05"}
Jul 01 19:26:46.526202 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.526087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.528787 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.528760    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:46.709755 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:46.709717    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:26:47.065869 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.065836    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 19:26:47.067313 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:26:47.067278    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df21efc_612d_4ab4_9723_0f12a1f83a52.slice/crio-d8fb3f9502b0842b9a8e15076d52dec090049dd07d9544d97c786e13f576e043 WatchSource:0}: Error finding container d8fb3f9502b0842b9a8e15076d52dec090049dd07d9544d97c786e13f576e043: Status 404 returned error can't find the container with id d8fb3f9502b0842b9a8e15076d52dec090049dd07d9544d97c786e13f576e043
Jul 01 19:26:47.266411 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.266369    2581 generic.go:356] "Generic (PLEG): container finished" podID="8476b125-8a53-4baa-b109-cb06bf728010" containerID="bcd8fac8b2a1fd626fe504dd75c191654c881114bab92adbc5dcb32492d9214d" exitCode=0
Jul 01 19:26:47.266598 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.266454    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x74m2" event={"ID":"8476b125-8a53-4baa-b109-cb06bf728010","Type":"ContainerDied","Data":"bcd8fac8b2a1fd626fe504dd75c191654c881114bab92adbc5dcb32492d9214d"}
Jul 01 19:26:47.267863 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.267841    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerStarted","Data":"d8fb3f9502b0842b9a8e15076d52dec090049dd07d9544d97c786e13f576e043"}
Jul 01 19:26:47.816899 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.816853    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6fc8754d75-psxk4"]
Jul 01 19:26:47.820340 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.820321    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:47.823211 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.823189    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-w4cl7\""
Jul 01 19:26:47.823308 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.823206    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 01 19:26:47.823308 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.823272    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 01 19:26:47.823423 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.823322    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 01 19:26:47.823423 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.823323    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 01 19:26:47.823529 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.823481    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5c0hlm1fbdn8u\""
Jul 01 19:26:47.824468 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.824449    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 01 19:26:47.833094 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.833073    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6fc8754d75-psxk4"]
Jul 01 19:26:47.941011 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.940974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:47.941163 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.941028    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-grpc-tls\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:47.941163 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.941097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:47.941278 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.941174    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:47.941278 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.941225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h4fn\" (UniqueName: \"kubernetes.io/projected/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-kube-api-access-9h4fn\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:47.941278 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.941257    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-tls\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:47.941426 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.941359    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:47.941426 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:47.941383    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-metrics-client-ca\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.042514 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.042469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.042514 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.042514    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-metrics-client-ca\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.042767 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.042603    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.042882 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.042785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-grpc-tls\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.042882 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.042845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.042988 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.042890    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.042988 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.042935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h4fn\" (UniqueName: \"kubernetes.io/projected/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-kube-api-access-9h4fn\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.042988 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.042980    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-tls\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.043497 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.043319    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-metrics-client-ca\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.046008 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.045961    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.046112 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.046006    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.046190 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.046114    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.046250 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.046194    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.046299 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.046256    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-thanos-querier-tls\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.046389 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.046359    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-secret-grpc-tls\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.058076 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.058052    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h4fn\" (UniqueName: \"kubernetes.io/projected/4ab58593-4ff0-41a5-a9f5-2c00f19d1c63-kube-api-access-9h4fn\") pod \"thanos-querier-6fc8754d75-psxk4\" (UID: \"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63\") " pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.130719 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.130628    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:48.273717 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.273687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x74m2" event={"ID":"8476b125-8a53-4baa-b109-cb06bf728010","Type":"ContainerStarted","Data":"184a4c6aa4a64a780651b98e332315fe5bfdb8c77782b2f7bb6e0efbd7322b24"}
Jul 01 19:26:48.273861 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.273752    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x74m2" event={"ID":"8476b125-8a53-4baa-b109-cb06bf728010","Type":"ContainerStarted","Data":"11f90c60111e7fab36e7fa56caa970c767cf65adb39d2c0141540191e2f388c0"}
Jul 01 19:26:48.297500 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.297211    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-x74m2" podStartSLOduration=2.614450038 podStartE2EDuration="4.297189069s" totalImagesPullingTime="1.614438421s" totalInitContainerRuntime="68.30061ms" isStatefulPod=true podCreationTimestamp="2026-07-01 19:26:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:48.296822959 +0000 UTC m=+319.531948272" watchObservedRunningTime="2026-07-01 19:26:48.297189069 +0000 UTC m=+319.532314382"
Jul 01 19:26:48.338502 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:48.338367    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6fc8754d75-psxk4"]
Jul 01 19:26:48.366364 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:26:48.366337    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab58593_4ff0_41a5_a9f5_2c00f19d1c63.slice/crio-3aa155836947146466486df9f5eb6210eb2a1624c1d0a76429ab43084f734591 WatchSource:0}: Error finding container 3aa155836947146466486df9f5eb6210eb2a1624c1d0a76429ab43084f734591: Status 404 returned error can't find the container with id 3aa155836947146466486df9f5eb6210eb2a1624c1d0a76429ab43084f734591
Jul 01 19:26:49.282496 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.282445    2581 generic.go:356] "Generic (PLEG): container finished" podID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerID="e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b" exitCode=0
Jul 01 19:26:49.282981 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.282606    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerDied","Data":"e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b"}
Jul 01 19:26:49.284807 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.284363    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4" event={"ID":"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63","Type":"ContainerStarted","Data":"3aa155836947146466486df9f5eb6210eb2a1624c1d0a76429ab43084f734591"}
Jul 01 19:26:49.385351 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.385307    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-69d4cb6966-ww6q2"]
Jul 01 19:26:49.406778 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.406735    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-69d4cb6966-ww6q2"]
Jul 01 19:26:49.406961 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.406854    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.411856 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.411826    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 01 19:26:49.412021 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.411899    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 01 19:26:49.412021 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.411923    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 01 19:26:49.412152 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.412050    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cvj0qsd3foad0\""
Jul 01 19:26:49.412152 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.412061    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-sxhfz\""
Jul 01 19:26:49.412320 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.412298    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 01 19:26:49.456845 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.456809    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-client-ca-bundle\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.457007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.456896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-secret-metrics-server-client-certs\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.457007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.456933    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-metrics-server-audit-profiles\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.457124 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.457036    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.457124 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.457094    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-secret-metrics-server-tls\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.457209 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.457182    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-audit-log\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.457309 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.457277    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rmlz\" (UniqueName: \"kubernetes.io/projected/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-kube-api-access-7rmlz\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.508902 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.508869    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"]
Jul 01 19:26:49.511673 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.511641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"
Jul 01 19:26:49.514859 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.514839    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-v4zck\""
Jul 01 19:26:49.515047 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.515033    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 01 19:26:49.522557 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.522537    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"]
Jul 01 19:26:49.558684 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.558614    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-secret-metrics-server-tls\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.558684 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.558674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-audit-log\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.558901 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.558709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rmlz\" (UniqueName: \"kubernetes.io/projected/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-kube-api-access-7rmlz\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.558901 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.558730    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/340a8dff-386b-4854-b97c-46a3f475e59b-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-ngt9m\" (UID: \"340a8dff-386b-4854-b97c-46a3f475e59b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"
Jul 01 19:26:49.558901 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.558758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfszt\" (UniqueName: \"kubernetes.io/projected/340a8dff-386b-4854-b97c-46a3f475e59b-kube-api-access-nfszt\") pod \"monitoring-plugin-cd8656b8-ngt9m\" (UID: \"340a8dff-386b-4854-b97c-46a3f475e59b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"
Jul 01 19:26:49.558901 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.558812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-client-ca-bundle\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.559089 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.559010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-secret-metrics-server-client-certs\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.559089 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.559043    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-metrics-server-audit-profiles\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.559186 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.559096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.559248 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.559230    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-audit-log\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.559814 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.559788    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.559937 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.559918    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-metrics-server-audit-profiles\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.561123 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.561103    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-client-ca-bundle\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.561199 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.561150    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-secret-metrics-server-tls\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.561245 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.561229    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-secret-metrics-server-client-certs\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.567060 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.567040    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rmlz\" (UniqueName: \"kubernetes.io/projected/02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d-kube-api-access-7rmlz\") pod \"metrics-server-69d4cb6966-ww6q2\" (UID: \"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d\") " pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.660299 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.660265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/340a8dff-386b-4854-b97c-46a3f475e59b-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-ngt9m\" (UID: \"340a8dff-386b-4854-b97c-46a3f475e59b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"
Jul 01 19:26:49.660475 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.660452    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfszt\" (UniqueName: \"kubernetes.io/projected/340a8dff-386b-4854-b97c-46a3f475e59b-kube-api-access-nfszt\") pod \"monitoring-plugin-cd8656b8-ngt9m\" (UID: \"340a8dff-386b-4854-b97c-46a3f475e59b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"
Jul 01 19:26:49.660546 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:49.660382    2581 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 01 19:26:49.660628 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:49.660601    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/340a8dff-386b-4854-b97c-46a3f475e59b-monitoring-plugin-cert podName:340a8dff-386b-4854-b97c-46a3f475e59b nodeName:}" failed. No retries permitted until 2026-07-01 19:26:50.160559556 +0000 UTC m=+321.395684869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/340a8dff-386b-4854-b97c-46a3f475e59b-monitoring-plugin-cert") pod "monitoring-plugin-cd8656b8-ngt9m" (UID: "340a8dff-386b-4854-b97c-46a3f475e59b") : secret "monitoring-plugin-cert" not found
Jul 01 19:26:49.669861 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.669829    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfszt\" (UniqueName: \"kubernetes.io/projected/340a8dff-386b-4854-b97c-46a3f475e59b-kube-api-access-nfszt\") pod \"monitoring-plugin-cd8656b8-ngt9m\" (UID: \"340a8dff-386b-4854-b97c-46a3f475e59b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"
Jul 01 19:26:49.719629 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.719595    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:26:49.858704 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:49.858618    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-69d4cb6966-ww6q2"]
Jul 01 19:26:49.861783 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:26:49.861729    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02b7019b_d5d1_4054_b8f6_4cbf27aa6e1d.slice/crio-5bf9e4e4f46254b5ad5b4b9e8d3b5e364f71e73cd430d2fc1151e4cc2948e223 WatchSource:0}: Error finding container 5bf9e4e4f46254b5ad5b4b9e8d3b5e364f71e73cd430d2fc1151e4cc2948e223: Status 404 returned error can't find the container with id 5bf9e4e4f46254b5ad5b4b9e8d3b5e364f71e73cd430d2fc1151e4cc2948e223
Jul 01 19:26:50.165158 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:50.165078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/340a8dff-386b-4854-b97c-46a3f475e59b-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-ngt9m\" (UID: \"340a8dff-386b-4854-b97c-46a3f475e59b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"
Jul 01 19:26:50.168076 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:50.168048    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/340a8dff-386b-4854-b97c-46a3f475e59b-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-ngt9m\" (UID: \"340a8dff-386b-4854-b97c-46a3f475e59b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"
Jul 01 19:26:50.290857 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:50.290810    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2" event={"ID":"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d","Type":"ContainerStarted","Data":"5bf9e4e4f46254b5ad5b4b9e8d3b5e364f71e73cd430d2fc1151e4cc2948e223"}
Jul 01 19:26:50.420736 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:50.420654    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"
Jul 01 19:26:50.577733 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:50.577688    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"]
Jul 01 19:26:51.237348 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:26:51.237310    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod340a8dff_386b_4854_b97c_46a3f475e59b.slice/crio-b8436801870365d39d3a7f2c04396a26d6288bb3537b79e23ea378d89ebe8773 WatchSource:0}: Error finding container b8436801870365d39d3a7f2c04396a26d6288bb3537b79e23ea378d89ebe8773: Status 404 returned error can't find the container with id b8436801870365d39d3a7f2c04396a26d6288bb3537b79e23ea378d89ebe8773
Jul 01 19:26:51.296297 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:51.296258    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m" event={"ID":"340a8dff-386b-4854-b97c-46a3f475e59b","Type":"ContainerStarted","Data":"b8436801870365d39d3a7f2c04396a26d6288bb3537b79e23ea378d89ebe8773"}
Jul 01 19:26:52.307054 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:52.306268    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2" event={"ID":"02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d","Type":"ContainerStarted","Data":"127d75b0503f7ae0633f7e0402c07b66c7e27375a1dcbda8819fcafad9876dec"}
Jul 01 19:26:52.310600 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:52.310400    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4" event={"ID":"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63","Type":"ContainerStarted","Data":"7e9da99cf03df2dac20618098c9ac324bbdc706be8a347022c3cb9f66854b46a"}
Jul 01 19:26:52.310600 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:52.310482    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4" event={"ID":"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63","Type":"ContainerStarted","Data":"0d9715456a058f25a39201bbf019bc2298a5d22522dab88c5cbd4b309180caa4"}
Jul 01 19:26:52.310600 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:52.310522    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4" event={"ID":"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63","Type":"ContainerStarted","Data":"f09c2e0bdd6cf01267b4483a86b8e3ad653d72da03c2dec6a208c13d59eb3c68"}
Jul 01 19:26:52.315169 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:52.315139    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerStarted","Data":"b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f"}
Jul 01 19:26:52.315301 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:52.315217    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerStarted","Data":"cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d"}
Jul 01 19:26:52.315301 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:52.315257    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerStarted","Data":"e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36"}
Jul 01 19:26:52.340958 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:52.340902    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2" podStartSLOduration=1.166341993 podStartE2EDuration="3.340884538s" totalImagesPullingTime="2.174542545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:52.340502767 +0000 UTC m=+323.575628080" watchObservedRunningTime="2026-07-01 19:26:52.340884538 +0000 UTC m=+323.576009853"
Jul 01 19:26:52.541788 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:52.541700    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67b8f5c454-d8tkg"]
Jul 01 19:26:52.542040 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:26:52.542016    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg" podUID="219b505f-ef8f-48a4-b1d8-ced51e82b7ba"
Jul 01 19:26:53.324027 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.323994    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerStarted","Data":"f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6"}
Jul 01 19:26:53.324027 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.324015    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:26:53.324491 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.324084    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerStarted","Data":"9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8"}
Jul 01 19:26:53.335959 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.335930    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:26:53.398603 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.398553    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-bound-sa-token\") pod \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") "
Jul 01 19:26:53.398603 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.398603    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg5h5\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-kube-api-access-dg5h5\") pod \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") "
Jul 01 19:26:53.398819 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.398639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-image-registry-private-configuration\") pod \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") "
Jul 01 19:26:53.398819 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.398678    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-trusted-ca\") pod \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") "
Jul 01 19:26:53.398819 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.398735    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-ca-trust-extracted\") pod \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") "
Jul 01 19:26:53.398819 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.398786    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-installation-pull-secrets\") pod \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") "
Jul 01 19:26:53.399006 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.398823    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-certificates\") pod \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\" (UID: \"219b505f-ef8f-48a4-b1d8-ced51e82b7ba\") "
Jul 01 19:26:53.399229 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.399200    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "219b505f-ef8f-48a4-b1d8-ced51e82b7ba" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:26:53.399357 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.399329    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "219b505f-ef8f-48a4-b1d8-ced51e82b7ba" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:26:53.399507 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.399450    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "219b505f-ef8f-48a4-b1d8-ced51e82b7ba" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:26:53.401439 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.401417    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "219b505f-ef8f-48a4-b1d8-ced51e82b7ba" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:26:53.401562 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.401530    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-kube-api-access-dg5h5" (OuterVolumeSpecName: "kube-api-access-dg5h5") pod "219b505f-ef8f-48a4-b1d8-ced51e82b7ba" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba"). InnerVolumeSpecName "kube-api-access-dg5h5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:26:53.401638 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.401554    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "219b505f-ef8f-48a4-b1d8-ced51e82b7ba" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:26:53.401831 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.401804    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "219b505f-ef8f-48a4-b1d8-ced51e82b7ba" (UID: "219b505f-ef8f-48a4-b1d8-ced51e82b7ba"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:26:53.500461 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.500435    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-installation-pull-secrets\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:26:53.500566 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.500465    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-certificates\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:26:53.500566 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.500480    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-bound-sa-token\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:26:53.500566 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.500494    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg5h5\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-kube-api-access-dg5h5\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:26:53.500566 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.500510    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-image-registry-private-configuration\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:26:53.500566 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.500525    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-trusted-ca\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:26:53.500566 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:53.500539    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-ca-trust-extracted\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:26:54.329779 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.329726    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m" event={"ID":"340a8dff-386b-4854-b97c-46a3f475e59b","Type":"ContainerStarted","Data":"719506a3dbaafd4a20c0cc5a22f3e93fd535d557c69f5f0473785f50b5bafc2e"}
Jul 01 19:26:54.330260 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.329800    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"
Jul 01 19:26:54.334199 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.334169    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4" event={"ID":"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63","Type":"ContainerStarted","Data":"df4e74a3f6e8f63ef30d96820b71fdc7c5886a6d79a042643beec8c4a6dc86a4"}
Jul 01 19:26:54.334359 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.334239    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4" event={"ID":"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63","Type":"ContainerStarted","Data":"3dcaaead2ec65034944d59d8b8906bd2a9c741e63a6e74605df3399f39e5a882"}
Jul 01 19:26:54.334359 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.334280    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4" event={"ID":"4ab58593-4ff0-41a5-a9f5-2c00f19d1c63","Type":"ContainerStarted","Data":"cd7b1aae3a4850fe493121b53994923444bc67443d1da2fed1ae80a54991f0cb"}
Jul 01 19:26:54.334359 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.334343    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:26:54.336311 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.336293    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m"
Jul 01 19:26:54.337824 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.337807    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b8f5c454-d8tkg"
Jul 01 19:26:54.337894 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.337837    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerStarted","Data":"0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26"}
Jul 01 19:26:54.352014 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.351959    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-ngt9m" podStartSLOduration=3.32031794 podStartE2EDuration="5.351937744s" totalImagesPullingTime="2.031619804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:54.35036507 +0000 UTC m=+325.585490383" watchObservedRunningTime="2026-07-01 19:26:54.351937744 +0000 UTC m=+325.587063057"
Jul 01 19:26:54.378435 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.378373    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4" podStartSLOduration=2.69640053 podStartE2EDuration="7.378355393s" totalImagesPullingTime="4.681954863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:26:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:54.375711676 +0000 UTC m=+325.610837011" watchObservedRunningTime="2026-07-01 19:26:54.378355393 +0000 UTC m=+325.613480732"
Jul 01 19:26:54.410261 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.410226    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67b8f5c454-d8tkg"]
Jul 01 19:26:54.415822 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.415791    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67b8f5c454-d8tkg"]
Jul 01 19:26:54.465349 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.465296    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.686770268 podStartE2EDuration="9.465282029s" totalImagesPullingTime="4.75855267s" totalInitContainerRuntime="19.959091ms" isStatefulPod=false podCreationTimestamp="2026-07-01 19:26:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:54.464840066 +0000 UTC m=+325.699965377" watchObservedRunningTime="2026-07-01 19:26:54.465282029 +0000 UTC m=+325.700407341"
Jul 01 19:26:54.512236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:54.512199    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219b505f-ef8f-48a4-b1d8-ced51e82b7ba-registry-tls\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:26:55.214563 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:26:55.214525    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="219b505f-ef8f-48a4-b1d8-ced51e82b7ba" path="/var/lib/kubelet/pods/219b505f-ef8f-48a4-b1d8-ced51e82b7ba/volumes"
Jul 01 19:27:00.350029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:00.349999    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6fc8754d75-psxk4"
Jul 01 19:27:09.719746 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.719699    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:27:09.720213 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.719779    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:27:09.748722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.748684    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7548cc4b47-gql59"]
Jul 01 19:27:09.815548 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.815511    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7548cc4b47-gql59"]
Jul 01 19:27:09.815726 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.815649    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:09.818842 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.818815    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 01 19:27:09.819045 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.819021    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 01 19:27:09.819120 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.819076    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 01 19:27:09.819120 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.819098    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 01 19:27:09.819228 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.819196    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 01 19:27:09.820348 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.820327    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 01 19:27:09.820457 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.820381    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rbgmq\""
Jul 01 19:27:09.820457 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.820429    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 01 19:27:09.951881 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.951849    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-serving-cert\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:09.952046 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.951913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-config\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:09.952046 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.951936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjfbl\" (UniqueName: \"kubernetes.io/projected/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-kube-api-access-mjfbl\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:09.952046 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.952009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-oauth-config\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:09.952198 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.952073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-oauth-serving-cert\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:09.952198 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:09.952129    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-service-ca\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.052966 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.052875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-oauth-config\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.052966 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.052940    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-oauth-serving-cert\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.052966 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.052964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-service-ca\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.053236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.053118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-serving-cert\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.053236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.053163    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-config\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.053236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.053182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjfbl\" (UniqueName: \"kubernetes.io/projected/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-kube-api-access-mjfbl\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.053681 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.053656    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-service-ca\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.053768 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.053661    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-oauth-serving-cert\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.053817 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.053800    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-config\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.055677 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.055654    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-serving-cert\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.055677 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.055668    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-oauth-config\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.063559 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.063534    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjfbl\" (UniqueName: \"kubernetes.io/projected/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-kube-api-access-mjfbl\") pod \"console-7548cc4b47-gql59\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") " pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.125872 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.125832    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:10.277932 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.277884    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7548cc4b47-gql59"]
Jul 01 19:27:10.280046 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:27:10.280018    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e75a1c_a6b4_479f_b0e1_0bd950efedbb.slice/crio-7a510413cf53ba59d14b5a8b5902e5ac305b2440557be80b89670b7c5df89471 WatchSource:0}: Error finding container 7a510413cf53ba59d14b5a8b5902e5ac305b2440557be80b89670b7c5df89471: Status 404 returned error can't find the container with id 7a510413cf53ba59d14b5a8b5902e5ac305b2440557be80b89670b7c5df89471
Jul 01 19:27:10.404352 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:10.404264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7548cc4b47-gql59" event={"ID":"95e75a1c-a6b4-479f-b0e1-0bd950efedbb","Type":"ContainerStarted","Data":"7a510413cf53ba59d14b5a8b5902e5ac305b2440557be80b89670b7c5df89471"}
Jul 01 19:27:13.420240 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:13.420148    2581 generic.go:356] "Generic (PLEG): container finished" podID="55638e79-3269-468e-a5c2-c8e1acd52a3d" containerID="c2a9e1ee0a0332008f85e020015062372cd671dc5671addedee3289f5eb9ea70" exitCode=0
Jul 01 19:27:13.420757 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:13.420231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb" event={"ID":"55638e79-3269-468e-a5c2-c8e1acd52a3d","Type":"ContainerDied","Data":"c2a9e1ee0a0332008f85e020015062372cd671dc5671addedee3289f5eb9ea70"}
Jul 01 19:27:13.420757 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:13.420681    2581 scope.go:117] "RemoveContainer" containerID="c2a9e1ee0a0332008f85e020015062372cd671dc5671addedee3289f5eb9ea70"
Jul 01 19:27:13.421857 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:13.421833    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7548cc4b47-gql59" event={"ID":"95e75a1c-a6b4-479f-b0e1-0bd950efedbb","Type":"ContainerStarted","Data":"72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71"}
Jul 01 19:27:13.463740 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:13.463690    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7548cc4b47-gql59" podStartSLOduration=1.598310018 podStartE2EDuration="4.463671986s" totalImagesPullingTime="2.865361968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:27:13.461749284 +0000 UTC m=+344.696874597" watchObservedRunningTime="2026-07-01 19:27:13.463671986 +0000 UTC m=+344.698797298"
Jul 01 19:27:14.426937 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:14.426904    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q8bfb" event={"ID":"55638e79-3269-468e-a5c2-c8e1acd52a3d","Type":"ContainerStarted","Data":"0ba96fa593d5d8f7600af6951efa47ec46e511f1ac733959ea7c4f6665a204a3"}
Jul 01 19:27:17.380973 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.380940    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dc9654d45-986nx"]
Jul 01 19:27:17.443442 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.443407    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dc9654d45-986nx"]
Jul 01 19:27:17.443693 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.443505    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.470675 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.470645    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 01 19:27:17.525360 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.525323    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb998592-faf1-4299-8bc4-0314e3b45329-console-serving-cert\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.525360 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.525360    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb998592-faf1-4299-8bc4-0314e3b45329-console-oauth-config\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.525635 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.525401    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-service-ca\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.525635 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.525444    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-console-config\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.525635 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.525463    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-oauth-serving-cert\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.525635 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.525479    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8522s\" (UniqueName: \"kubernetes.io/projected/cb998592-faf1-4299-8bc4-0314e3b45329-kube-api-access-8522s\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.525635 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.525515    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-trusted-ca-bundle\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.626276 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.626231    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-trusted-ca-bundle\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.626477 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.626336    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb998592-faf1-4299-8bc4-0314e3b45329-console-serving-cert\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.626477 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.626367    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb998592-faf1-4299-8bc4-0314e3b45329-console-oauth-config\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.626477 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.626428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-service-ca\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.626477 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.626469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-console-config\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.626734 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.626496    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-oauth-serving-cert\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.626734 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.626523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8522s\" (UniqueName: \"kubernetes.io/projected/cb998592-faf1-4299-8bc4-0314e3b45329-kube-api-access-8522s\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.627241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.627211    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-service-ca\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.627241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.627235    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-console-config\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.627377 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.627260    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-oauth-serving-cert\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.627415 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.627401    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-trusted-ca-bundle\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.628840 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.628817    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb998592-faf1-4299-8bc4-0314e3b45329-console-oauth-config\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.629019 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.628999    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb998592-faf1-4299-8bc4-0314e3b45329-console-serving-cert\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.639233 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.639175    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8522s\" (UniqueName: \"kubernetes.io/projected/cb998592-faf1-4299-8bc4-0314e3b45329-kube-api-access-8522s\") pod \"console-7dc9654d45-986nx\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") " pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.753458 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.753424    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:17.892692 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:17.892651    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dc9654d45-986nx"]
Jul 01 19:27:17.894677 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:27:17.894652    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb998592_faf1_4299_8bc4_0314e3b45329.slice/crio-599c589df4df31cd512c51b91a27641509d1d2a0b8bc37baa3b13d9949084a4b WatchSource:0}: Error finding container 599c589df4df31cd512c51b91a27641509d1d2a0b8bc37baa3b13d9949084a4b: Status 404 returned error can't find the container with id 599c589df4df31cd512c51b91a27641509d1d2a0b8bc37baa3b13d9949084a4b
Jul 01 19:27:18.444108 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:18.444069    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc9654d45-986nx" event={"ID":"cb998592-faf1-4299-8bc4-0314e3b45329","Type":"ContainerStarted","Data":"c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c"}
Jul 01 19:27:18.444499 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:18.444153    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc9654d45-986nx" event={"ID":"cb998592-faf1-4299-8bc4-0314e3b45329","Type":"ContainerStarted","Data":"599c589df4df31cd512c51b91a27641509d1d2a0b8bc37baa3b13d9949084a4b"}
Jul 01 19:27:18.473407 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:18.473348    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dc9654d45-986nx" podStartSLOduration=1.473327357 podStartE2EDuration="1.473327357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:27:18.470103027 +0000 UTC m=+349.705228339" watchObservedRunningTime="2026-07-01 19:27:18.473327357 +0000 UTC m=+349.708452670"
Jul 01 19:27:20.126473 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:20.126436    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:20.126892 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:20.126493    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:20.131210 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:20.131188    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:20.454970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:20.454870    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:27.480439 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:27.480408    2581 generic.go:356] "Generic (PLEG): container finished" podID="5f011f0f-0e35-4018-8fb0-08180a66a798" containerID="96e56ee62ad8b3860ce98bb238dd42bf8fa82e68e79b2b4c811b030e18038192" exitCode=0
Jul 01 19:27:27.480889 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:27.480485    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr" event={"ID":"5f011f0f-0e35-4018-8fb0-08180a66a798","Type":"ContainerDied","Data":"96e56ee62ad8b3860ce98bb238dd42bf8fa82e68e79b2b4c811b030e18038192"}
Jul 01 19:27:27.480889 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:27.480835    2581 scope.go:117] "RemoveContainer" containerID="96e56ee62ad8b3860ce98bb238dd42bf8fa82e68e79b2b4c811b030e18038192"
Jul 01 19:27:27.754479 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:27.754377    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:27.754479 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:27.754450    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:27.760845 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:27.760813    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:28.491991 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:28.491955    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-crtfr" event={"ID":"5f011f0f-0e35-4018-8fb0-08180a66a798","Type":"ContainerStarted","Data":"6b48dc82e409458635c9ce63d99b3d1d9e8003b1d4ad4ae7944503c39b42a103"}
Jul 01 19:27:28.496003 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:28.495977    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:27:28.613762 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:28.613728    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7548cc4b47-gql59"]
Jul 01 19:27:29.724975 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:29.724945    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:27:29.729044 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:29.729016    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-69d4cb6966-ww6q2"
Jul 01 19:27:34.515209 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:34.515173    2581 generic.go:356] "Generic (PLEG): container finished" podID="5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40" containerID="692db34d94af4ee9520c621377db0ed5663cb415c75cad097853f328fb467d94" exitCode=0
Jul 01 19:27:34.515597 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:34.515247    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-gmpj2" event={"ID":"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40","Type":"ContainerDied","Data":"692db34d94af4ee9520c621377db0ed5663cb415c75cad097853f328fb467d94"}
Jul 01 19:27:34.515643 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:34.515625    2581 scope.go:117] "RemoveContainer" containerID="692db34d94af4ee9520c621377db0ed5663cb415c75cad097853f328fb467d94"
Jul 01 19:27:35.521251 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:35.521219    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-gmpj2" event={"ID":"5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40","Type":"ContainerStarted","Data":"f19003c5a0496b2a932cf5c877101a13bba5600d53ed8564ab38fa35fed5a799"}
Jul 01 19:27:43.264380 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:43.264343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:27:43.266611 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:43.266594    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/161462f4-ad95-4e5a-ab54-148c05190926-metrics-certs\") pod \"network-metrics-daemon-lw425\" (UID: \"161462f4-ad95-4e5a-ab54-148c05190926\") " pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:27:43.514056 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:43.514024    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tjdnw\""
Jul 01 19:27:43.521821 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:43.521773    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw425"
Jul 01 19:27:43.653024 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:43.653002    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lw425"]
Jul 01 19:27:43.655276 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:27:43.655248    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod161462f4_ad95_4e5a_ab54_148c05190926.slice/crio-4dc0bfb125b2f0592d4c0e4f92f26ea3e6d3feaa88dd8430e4a39f32cb6c4cfe WatchSource:0}: Error finding container 4dc0bfb125b2f0592d4c0e4f92f26ea3e6d3feaa88dd8430e4a39f32cb6c4cfe: Status 404 returned error can't find the container with id 4dc0bfb125b2f0592d4c0e4f92f26ea3e6d3feaa88dd8430e4a39f32cb6c4cfe
Jul 01 19:27:44.558658 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:44.558616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lw425" event={"ID":"161462f4-ad95-4e5a-ab54-148c05190926","Type":"ContainerStarted","Data":"4dc0bfb125b2f0592d4c0e4f92f26ea3e6d3feaa88dd8430e4a39f32cb6c4cfe"}
Jul 01 19:27:45.564930 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:45.564895    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lw425" event={"ID":"161462f4-ad95-4e5a-ab54-148c05190926","Type":"ContainerStarted","Data":"1467486bcf809ff218d947668ade563851d50b65641f9184bb18d188086c45b1"}
Jul 01 19:27:45.565333 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:45.564951    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lw425" event={"ID":"161462f4-ad95-4e5a-ab54-148c05190926","Type":"ContainerStarted","Data":"8c9713e82d07db323c66fa30c7413571a1d7c69ed59ce2e22a9c7c1a126b1d6a"}
Jul 01 19:27:45.584460 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:45.584407    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lw425" podStartSLOduration=375.537624166 podStartE2EDuration="6m16.584389657s" totalImagesPullingTime="1.046765491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:27:45.58266066 +0000 UTC m=+376.817785973" watchObservedRunningTime="2026-07-01 19:27:45.584389657 +0000 UTC m=+376.819514969"
Jul 01 19:27:53.639978 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:53.639912    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7548cc4b47-gql59" podUID="95e75a1c-a6b4-479f-b0e1-0bd950efedbb" containerName="console" containerID="cri-o://72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71" gracePeriod=15
Jul 01 19:27:53.902075 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:53.902004    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:54.062532 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.062498    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjfbl\" (UniqueName: \"kubernetes.io/projected/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-kube-api-access-mjfbl\") pod \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") "
Jul 01 19:27:54.062756 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.062597    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-service-ca\") pod \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") "
Jul 01 19:27:54.062756 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.062636    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-oauth-config\") pod \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") "
Jul 01 19:27:54.062756 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.062660    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-oauth-serving-cert\") pod \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") "
Jul 01 19:27:54.062756 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.062694    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-serving-cert\") pod \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") "
Jul 01 19:27:54.062756 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.062755    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-config\") pod \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\" (UID: \"95e75a1c-a6b4-479f-b0e1-0bd950efedbb\") "
Jul 01 19:27:54.063039 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.063009    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-service-ca" (OuterVolumeSpecName: "service-ca") pod "95e75a1c-a6b4-479f-b0e1-0bd950efedbb" (UID: "95e75a1c-a6b4-479f-b0e1-0bd950efedbb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:27:54.063165 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.063136    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "95e75a1c-a6b4-479f-b0e1-0bd950efedbb" (UID: "95e75a1c-a6b4-479f-b0e1-0bd950efedbb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:27:54.063270 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.063159    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-config" (OuterVolumeSpecName: "console-config") pod "95e75a1c-a6b4-479f-b0e1-0bd950efedbb" (UID: "95e75a1c-a6b4-479f-b0e1-0bd950efedbb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:27:54.065003 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.064977    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "95e75a1c-a6b4-479f-b0e1-0bd950efedbb" (UID: "95e75a1c-a6b4-479f-b0e1-0bd950efedbb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:27:54.065092 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.065002    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-kube-api-access-mjfbl" (OuterVolumeSpecName: "kube-api-access-mjfbl") pod "95e75a1c-a6b4-479f-b0e1-0bd950efedbb" (UID: "95e75a1c-a6b4-479f-b0e1-0bd950efedbb"). InnerVolumeSpecName "kube-api-access-mjfbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:27:54.065092 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.065024    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "95e75a1c-a6b4-479f-b0e1-0bd950efedbb" (UID: "95e75a1c-a6b4-479f-b0e1-0bd950efedbb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:27:54.163805 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.163714    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-service-ca\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:27:54.163805 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.163746    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-oauth-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:27:54.163805 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.163756    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-oauth-serving-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:27:54.163805 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.163765    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-serving-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:27:54.163805 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.163774    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-console-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:27:54.163805 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.163781    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjfbl\" (UniqueName: \"kubernetes.io/projected/95e75a1c-a6b4-479f-b0e1-0bd950efedbb-kube-api-access-mjfbl\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:27:54.605592 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.605551    2581 generic.go:356] "Generic (PLEG): container finished" podID="95e75a1c-a6b4-479f-b0e1-0bd950efedbb" containerID="72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71" exitCode=0
Jul 01 19:27:54.605765 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.605614    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7548cc4b47-gql59"
Jul 01 19:27:54.605765 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.605621    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7548cc4b47-gql59" event={"ID":"95e75a1c-a6b4-479f-b0e1-0bd950efedbb","Type":"ContainerDied","Data":"72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71"}
Jul 01 19:27:54.605765 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.605661    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7548cc4b47-gql59" event={"ID":"95e75a1c-a6b4-479f-b0e1-0bd950efedbb","Type":"ContainerDied","Data":"7a510413cf53ba59d14b5a8b5902e5ac305b2440557be80b89670b7c5df89471"}
Jul 01 19:27:54.605765 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.605693    2581 scope.go:117] "RemoveContainer" containerID="72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71"
Jul 01 19:27:54.614716 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.614695    2581 scope.go:117] "RemoveContainer" containerID="72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71"
Jul 01 19:27:54.614991 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:27:54.614973    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71\": container with ID starting with 72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71 not found: ID does not exist" containerID="72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71"
Jul 01 19:27:54.615052 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.615000    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71"} err="failed to get container status \"72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71\": rpc error: code = NotFound desc = could not find container \"72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71\": container with ID starting with 72b24528303c3a341a2c1e4485c913b7821980404fa35c23b82cdc7ec7df6c71 not found: ID does not exist"
Jul 01 19:27:54.627876 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.627844    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7548cc4b47-gql59"]
Jul 01 19:27:54.635097 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:54.635073    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7548cc4b47-gql59"]
Jul 01 19:27:55.214800 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:27:55.214767    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95e75a1c-a6b4-479f-b0e1-0bd950efedbb" path="/var/lib/kubelet/pods/95e75a1c-a6b4-479f-b0e1-0bd950efedbb/volumes"
Jul 01 19:28:05.295594 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.295539    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 19:28:05.296217 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.296182    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="kube-rbac-proxy-web" containerID="cri-o://b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f" gracePeriod=120
Jul 01 19:28:05.296364 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.296287    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="prom-label-proxy" containerID="cri-o://0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26" gracePeriod=120
Jul 01 19:28:05.296364 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.296179    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="config-reloader" containerID="cri-o://cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d" gracePeriod=120
Jul 01 19:28:05.296488 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.296187    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="kube-rbac-proxy-metric" containerID="cri-o://f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6" gracePeriod=120
Jul 01 19:28:05.296488 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.296174    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="kube-rbac-proxy" containerID="cri-o://9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8" gracePeriod=120
Jul 01 19:28:05.296488 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.296226    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="alertmanager" containerID="cri-o://e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36" gracePeriod=120
Jul 01 19:28:05.653704 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.653604    2581 generic.go:356] "Generic (PLEG): container finished" podID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerID="0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26" exitCode=0
Jul 01 19:28:05.653704 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.653633    2581 generic.go:356] "Generic (PLEG): container finished" podID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerID="9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8" exitCode=0
Jul 01 19:28:05.653704 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.653642    2581 generic.go:356] "Generic (PLEG): container finished" podID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerID="cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d" exitCode=0
Jul 01 19:28:05.653704 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.653650    2581 generic.go:356] "Generic (PLEG): container finished" podID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerID="e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36" exitCode=0
Jul 01 19:28:05.653704 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.653679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerDied","Data":"0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26"}
Jul 01 19:28:05.653987 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.653751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerDied","Data":"9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8"}
Jul 01 19:28:05.653987 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.653805    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerDied","Data":"cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d"}
Jul 01 19:28:05.653987 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:05.653833    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerDied","Data":"e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36"}
Jul 01 19:28:06.542180 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.542153    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:06.660926 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.660834    2581 generic.go:356] "Generic (PLEG): container finished" podID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerID="f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6" exitCode=0
Jul 01 19:28:06.660926 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.660870    2581 generic.go:356] "Generic (PLEG): container finished" podID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerID="b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f" exitCode=0
Jul 01 19:28:06.660926 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.660913    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerDied","Data":"f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6"}
Jul 01 19:28:06.661142 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.660933    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:06.661142 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.660983    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerDied","Data":"b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f"}
Jul 01 19:28:06.661142 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.661014    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2df21efc-612d-4ab4-9723-0f12a1f83a52","Type":"ContainerDied","Data":"d8fb3f9502b0842b9a8e15076d52dec090049dd07d9544d97c786e13f576e043"}
Jul 01 19:28:06.661142 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.661045    2581 scope.go:117] "RemoveContainer" containerID="0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26"
Jul 01 19:28:06.668589 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.668554    2581 scope.go:117] "RemoveContainer" containerID="f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6"
Jul 01 19:28:06.675997 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.675974    2581 scope.go:117] "RemoveContainer" containerID="9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8"
Jul 01 19:28:06.679140 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679121    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy-metric\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679221 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679162    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2df21efc-612d-4ab4-9723-0f12a1f83a52-alertmanager-main-db\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679221 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679192    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-web-config\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679221 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679218    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-config-volume\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679384 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679249    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2df21efc-612d-4ab4-9723-0f12a1f83a52-alertmanager-trusted-ca-bundle\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679384 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679290    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2df21efc-612d-4ab4-9723-0f12a1f83a52-config-out\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679384 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679316    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy-web\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679384 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679351    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2df21efc-612d-4ab4-9723-0f12a1f83a52-metrics-client-ca\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679816 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679389    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-main-tls\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679816 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679416    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2df21efc-612d-4ab4-9723-0f12a1f83a52-tls-assets\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679816 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679439    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679816 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679492    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-cluster-tls-config\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679816 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679520    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wfzw\" (UniqueName: \"kubernetes.io/projected/2df21efc-612d-4ab4-9723-0f12a1f83a52-kube-api-access-7wfzw\") pod \"2df21efc-612d-4ab4-9723-0f12a1f83a52\" (UID: \"2df21efc-612d-4ab4-9723-0f12a1f83a52\") "
Jul 01 19:28:06.679816 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679700    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2df21efc-612d-4ab4-9723-0f12a1f83a52-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:28:06.680146 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679449    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2df21efc-612d-4ab4-9723-0f12a1f83a52-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:28:06.680146 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679837    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2df21efc-612d-4ab4-9723-0f12a1f83a52-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:28:06.680146 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.679865    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2df21efc-612d-4ab4-9723-0f12a1f83a52-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.683005 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.682456    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:28:06.683179 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.683147    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-config-volume" (OuterVolumeSpecName: "config-volume") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:28:06.683985 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.683901    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:28:06.683985 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.683911    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2df21efc-612d-4ab4-9723-0f12a1f83a52-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:28:06.683985 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.683919    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2df21efc-612d-4ab4-9723-0f12a1f83a52-kube-api-access-7wfzw" (OuterVolumeSpecName: "kube-api-access-7wfzw") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "kube-api-access-7wfzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:28:06.683985 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.683962    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2df21efc-612d-4ab4-9723-0f12a1f83a52-config-out" (OuterVolumeSpecName: "config-out") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:28:06.684234 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.684008    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:28:06.685834 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.685804    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:28:06.686702 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.686682    2581 scope.go:117] "RemoveContainer" containerID="b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f"
Jul 01 19:28:06.687205 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.687182    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:28:06.695051 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.695017    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-web-config" (OuterVolumeSpecName: "web-config") pod "2df21efc-612d-4ab4-9723-0f12a1f83a52" (UID: "2df21efc-612d-4ab4-9723-0f12a1f83a52"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:28:06.695763 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.695745    2581 scope.go:117] "RemoveContainer" containerID="cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d"
Jul 01 19:28:06.703225 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.703209    2581 scope.go:117] "RemoveContainer" containerID="e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36"
Jul 01 19:28:06.712016 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.711997    2581 scope.go:117] "RemoveContainer" containerID="e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b"
Jul 01 19:28:06.719220 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.719200    2581 scope.go:117] "RemoveContainer" containerID="0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26"
Jul 01 19:28:06.719493 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:28:06.719473    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26\": container with ID starting with 0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26 not found: ID does not exist" containerID="0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26"
Jul 01 19:28:06.719543 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.719503    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26"} err="failed to get container status \"0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26\": rpc error: code = NotFound desc = could not find container \"0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26\": container with ID starting with 0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26 not found: ID does not exist"
Jul 01 19:28:06.719543 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.719524    2581 scope.go:117] "RemoveContainer" containerID="f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6"
Jul 01 19:28:06.719803 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:28:06.719784    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6\": container with ID starting with f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6 not found: ID does not exist" containerID="f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6"
Jul 01 19:28:06.719851 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.719810    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6"} err="failed to get container status \"f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6\": rpc error: code = NotFound desc = could not find container \"f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6\": container with ID starting with f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6 not found: ID does not exist"
Jul 01 19:28:06.719851 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.719829    2581 scope.go:117] "RemoveContainer" containerID="9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8"
Jul 01 19:28:06.720097 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:28:06.720070    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8\": container with ID starting with 9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8 not found: ID does not exist" containerID="9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8"
Jul 01 19:28:06.720164 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.720104    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8"} err="failed to get container status \"9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8\": rpc error: code = NotFound desc = could not find container \"9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8\": container with ID starting with 9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8 not found: ID does not exist"
Jul 01 19:28:06.720164 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.720127    2581 scope.go:117] "RemoveContainer" containerID="b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f"
Jul 01 19:28:06.720384 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:28:06.720367    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f\": container with ID starting with b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f not found: ID does not exist" containerID="b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f"
Jul 01 19:28:06.720428 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.720388    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f"} err="failed to get container status \"b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f\": rpc error: code = NotFound desc = could not find container \"b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f\": container with ID starting with b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f not found: ID does not exist"
Jul 01 19:28:06.720428 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.720402    2581 scope.go:117] "RemoveContainer" containerID="cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d"
Jul 01 19:28:06.720658 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:28:06.720640    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d\": container with ID starting with cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d not found: ID does not exist" containerID="cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d"
Jul 01 19:28:06.720721 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.720665    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d"} err="failed to get container status \"cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d\": rpc error: code = NotFound desc = could not find container \"cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d\": container with ID starting with cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d not found: ID does not exist"
Jul 01 19:28:06.720721 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.720684    2581 scope.go:117] "RemoveContainer" containerID="e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36"
Jul 01 19:28:06.720926 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:28:06.720910    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36\": container with ID starting with e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36 not found: ID does not exist" containerID="e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36"
Jul 01 19:28:06.720960 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.720930    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36"} err="failed to get container status \"e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36\": rpc error: code = NotFound desc = could not find container \"e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36\": container with ID starting with e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36 not found: ID does not exist"
Jul 01 19:28:06.720960 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.720945    2581 scope.go:117] "RemoveContainer" containerID="e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b"
Jul 01 19:28:06.721140 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:28:06.721122    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b\": container with ID starting with e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b not found: ID does not exist" containerID="e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b"
Jul 01 19:28:06.721179 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.721145    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b"} err="failed to get container status \"e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b\": rpc error: code = NotFound desc = could not find container \"e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b\": container with ID starting with e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b not found: ID does not exist"
Jul 01 19:28:06.721179 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.721158    2581 scope.go:117] "RemoveContainer" containerID="0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26"
Jul 01 19:28:06.721360 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.721346    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26"} err="failed to get container status \"0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26\": rpc error: code = NotFound desc = could not find container \"0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26\": container with ID starting with 0fb027e1c643bef8859e347f8cf8a1b35dd8f0edf8533718e1073833db81ab26 not found: ID does not exist"
Jul 01 19:28:06.721405 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.721360    2581 scope.go:117] "RemoveContainer" containerID="f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6"
Jul 01 19:28:06.721558 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.721542    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6"} err="failed to get container status \"f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6\": rpc error: code = NotFound desc = could not find container \"f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6\": container with ID starting with f7f73dd3e15614f38df24647ede70b67da38deb9d8766f024c3a073ddc51a0e6 not found: ID does not exist"
Jul 01 19:28:06.721625 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.721560    2581 scope.go:117] "RemoveContainer" containerID="9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8"
Jul 01 19:28:06.721789 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.721767    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8"} err="failed to get container status \"9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8\": rpc error: code = NotFound desc = could not find container \"9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8\": container with ID starting with 9f4b8fa995aadac61c97338cbcb121ead53b947aef8a4c394ea38f3d200fb1f8 not found: ID does not exist"
Jul 01 19:28:06.721828 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.721789    2581 scope.go:117] "RemoveContainer" containerID="b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f"
Jul 01 19:28:06.721991 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.721972    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f"} err="failed to get container status \"b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f\": rpc error: code = NotFound desc = could not find container \"b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f\": container with ID starting with b3b21431e6598618a473c9e1d375d9f8bf5ae129a896ba60af33d9989c5cdd5f not found: ID does not exist"
Jul 01 19:28:06.722059 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.721993    2581 scope.go:117] "RemoveContainer" containerID="cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d"
Jul 01 19:28:06.722218 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.722201    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d"} err="failed to get container status \"cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d\": rpc error: code = NotFound desc = could not find container \"cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d\": container with ID starting with cef1fec92f3c630323f6567f6fcb1612bd493f6c4937f6148d6382a8fbb1652d not found: ID does not exist"
Jul 01 19:28:06.722259 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.722219    2581 scope.go:117] "RemoveContainer" containerID="e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36"
Jul 01 19:28:06.722419 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.722402    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36"} err="failed to get container status \"e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36\": rpc error: code = NotFound desc = could not find container \"e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36\": container with ID starting with e97ab63de36452d5304730e31f49de77cc04d05099f497e6df85373f60241b36 not found: ID does not exist"
Jul 01 19:28:06.722485 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.722420    2581 scope.go:117] "RemoveContainer" containerID="e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b"
Jul 01 19:28:06.722645 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.722628    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b"} err="failed to get container status \"e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b\": rpc error: code = NotFound desc = could not find container \"e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b\": container with ID starting with e134a43b4466fe4cb7734242959473669b47fa15efa64a40b828f2f95fdba68b not found: ID does not exist"
Jul 01 19:28:06.780793 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.780758    2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-cluster-tls-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.780793 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.780787    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wfzw\" (UniqueName: \"kubernetes.io/projected/2df21efc-612d-4ab4-9723-0f12a1f83a52-kube-api-access-7wfzw\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.780793 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.780797    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.780970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.780806    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2df21efc-612d-4ab4-9723-0f12a1f83a52-alertmanager-main-db\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.780970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.780816    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-web-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.780970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.780824    2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-config-volume\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.780970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.780832    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2df21efc-612d-4ab4-9723-0f12a1f83a52-config-out\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.780970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.780840    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.780970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.780849    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2df21efc-612d-4ab4-9723-0f12a1f83a52-metrics-client-ca\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.780970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.780857    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-main-tls\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.780970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.780865    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2df21efc-612d-4ab4-9723-0f12a1f83a52-tls-assets\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.780970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.780873    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2df21efc-612d-4ab4-9723-0f12a1f83a52-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:06.990756 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:06.990722    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 19:28:07.000897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.000862    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 19:28:07.037082 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037047    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 19:28:07.037377 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037364    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="init-config-reloader"
Jul 01 19:28:07.037419 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037380    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="init-config-reloader"
Jul 01 19:28:07.037419 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037396    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="kube-rbac-proxy-metric"
Jul 01 19:28:07.037419 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037402    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="kube-rbac-proxy-metric"
Jul 01 19:28:07.037419 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037416    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="alertmanager"
Jul 01 19:28:07.037551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037421    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="alertmanager"
Jul 01 19:28:07.037551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037428    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95e75a1c-a6b4-479f-b0e1-0bd950efedbb" containerName="console"
Jul 01 19:28:07.037551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037433    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="95e75a1c-a6b4-479f-b0e1-0bd950efedbb" containerName="console"
Jul 01 19:28:07.037551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037441    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="kube-rbac-proxy"
Jul 01 19:28:07.037551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037446    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="kube-rbac-proxy"
Jul 01 19:28:07.037551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037452    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="prom-label-proxy"
Jul 01 19:28:07.037551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037459    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="prom-label-proxy"
Jul 01 19:28:07.037551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037476    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="config-reloader"
Jul 01 19:28:07.037551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037483    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="config-reloader"
Jul 01 19:28:07.037551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037488    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="kube-rbac-proxy-web"
Jul 01 19:28:07.037551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037493    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="kube-rbac-proxy-web"
Jul 01 19:28:07.037551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037551    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="kube-rbac-proxy-metric"
Jul 01 19:28:07.037939 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037562    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="prom-label-proxy"
Jul 01 19:28:07.037939 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037568    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="95e75a1c-a6b4-479f-b0e1-0bd950efedbb" containerName="console"
Jul 01 19:28:07.037939 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037587    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="alertmanager"
Jul 01 19:28:07.037939 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037594    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="config-reloader"
Jul 01 19:28:07.037939 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037599    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="kube-rbac-proxy-web"
Jul 01 19:28:07.037939 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.037605    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" containerName="kube-rbac-proxy"
Jul 01 19:28:07.042509 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.042489    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.047358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.047327    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 01 19:28:07.047506 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.047343    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 01 19:28:07.047506 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.047393    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 01 19:28:07.047506 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.047426    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 01 19:28:07.047506 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.047426    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 01 19:28:07.047820 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.047632    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 01 19:28:07.047820 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.047708    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 01 19:28:07.047820 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.047761    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-cl297\""
Jul 01 19:28:07.047978 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.047852    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 01 19:28:07.056619 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.056571    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 01 19:28:07.082358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.082330    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 19:28:07.184915 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.184877    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55749f0c-bc29-49cb-9346-9687d3179626-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.184915 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.184913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.185132 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.184938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.185132 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.184966    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55749f0c-bc29-49cb-9346-9687d3179626-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.185132 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.184999    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-config-volume\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.185132 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.185018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55749f0c-bc29-49cb-9346-9687d3179626-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.185132 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.185085    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55749f0c-bc29-49cb-9346-9687d3179626-config-out\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.185132 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.185117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fdrh\" (UniqueName: \"kubernetes.io/projected/55749f0c-bc29-49cb-9346-9687d3179626-kube-api-access-5fdrh\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.185382 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.185165    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.185382 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.185186    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55749f0c-bc29-49cb-9346-9687d3179626-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.185382 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.185220    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.185382 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.185235    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.185382 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.185277    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-web-config\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.214878 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.214847    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2df21efc-612d-4ab4-9723-0f12a1f83a52" path="/var/lib/kubelet/pods/2df21efc-612d-4ab4-9723-0f12a1f83a52/volumes"
Jul 01 19:28:07.286429 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55749f0c-bc29-49cb-9346-9687d3179626-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.286429 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.286429 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286409    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.286429 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55749f0c-bc29-49cb-9346-9687d3179626-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.286773 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286485    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-config-volume\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.286773 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55749f0c-bc29-49cb-9346-9687d3179626-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.286773 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55749f0c-bc29-49cb-9346-9687d3179626-config-out\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.286773 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fdrh\" (UniqueName: \"kubernetes.io/projected/55749f0c-bc29-49cb-9346-9687d3179626-kube-api-access-5fdrh\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.286773 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286724    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.286773 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55749f0c-bc29-49cb-9346-9687d3179626-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.287070 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.287070 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.287070 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286885    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-web-config\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.287070 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.286918    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/55749f0c-bc29-49cb-9346-9687d3179626-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.287875 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.287512    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55749f0c-bc29-49cb-9346-9687d3179626-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.287875 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.287534    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55749f0c-bc29-49cb-9346-9687d3179626-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.289847 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.289683    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/55749f0c-bc29-49cb-9346-9687d3179626-config-out\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.289847 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.289806    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.289847 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.289835    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-config-volume\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.290005 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.289903    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-web-config\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.290111 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.290093    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.290146 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.290123    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.290180 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.290142    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.291141 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.291126    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/55749f0c-bc29-49cb-9346-9687d3179626-tls-assets\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.291699 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.291683    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/55749f0c-bc29-49cb-9346-9687d3179626-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.301333 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.301304    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fdrh\" (UniqueName: \"kubernetes.io/projected/55749f0c-bc29-49cb-9346-9687d3179626-kube-api-access-5fdrh\") pod \"alertmanager-main-0\" (UID: \"55749f0c-bc29-49cb-9346-9687d3179626\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.353220 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.353190    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 19:28:07.490891 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.490743    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 19:28:07.493251 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:28:07.493225    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55749f0c_bc29_49cb_9346_9687d3179626.slice/crio-dffa284249036a926ee6b08710109233ae9441d36b514d8cbe32ba3fa6cbe0e5 WatchSource:0}: Error finding container dffa284249036a926ee6b08710109233ae9441d36b514d8cbe32ba3fa6cbe0e5: Status 404 returned error can't find the container with id dffa284249036a926ee6b08710109233ae9441d36b514d8cbe32ba3fa6cbe0e5
Jul 01 19:28:07.666632 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.666592    2581 generic.go:356] "Generic (PLEG): container finished" podID="55749f0c-bc29-49cb-9346-9687d3179626" containerID="f03fc48f7d275b917241850ebb8b03a3f12636aca33c100b57ff7d31e4c0962b" exitCode=0
Jul 01 19:28:07.667059 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.666686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55749f0c-bc29-49cb-9346-9687d3179626","Type":"ContainerDied","Data":"f03fc48f7d275b917241850ebb8b03a3f12636aca33c100b57ff7d31e4c0962b"}
Jul 01 19:28:07.667059 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.666789    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55749f0c-bc29-49cb-9346-9687d3179626","Type":"ContainerStarted","Data":"dffa284249036a926ee6b08710109233ae9441d36b514d8cbe32ba3fa6cbe0e5"}
Jul 01 19:28:07.674536 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.674508    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-587867dc88-vzb79"]
Jul 01 19:28:07.678164 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.678143    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.692705 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.692551    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-oauth-serving-cert\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.692705 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.692683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-config\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.692869 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.692778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-serving-cert\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.693613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.693025    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-oauth-config\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.693613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.693097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt649\" (UniqueName: \"kubernetes.io/projected/6f665472-c2bd-48fb-b52b-a3597a9e521a-kube-api-access-nt649\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.693613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.693236    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-trusted-ca-bundle\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.693613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.693274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-service-ca\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.694256 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.694235    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-587867dc88-vzb79"]
Jul 01 19:28:07.794222 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.794199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-oauth-serving-cert\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.794312 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.794243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-config\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.794312 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.794269    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-serving-cert\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.794312 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.794297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-oauth-config\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.794427 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.794315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt649\" (UniqueName: \"kubernetes.io/projected/6f665472-c2bd-48fb-b52b-a3597a9e521a-kube-api-access-nt649\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.794427 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.794340    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-trusted-ca-bundle\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.794427 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.794356    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-service-ca\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.795254 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.795231    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-service-ca\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.795362 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.795234    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-config\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.795362 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.795335    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-oauth-serving-cert\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.795535 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.795446    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-trusted-ca-bundle\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.797221 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.797191    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-serving-cert\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.797920 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.797871    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-oauth-config\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.803245 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.803182    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt649\" (UniqueName: \"kubernetes.io/projected/6f665472-c2bd-48fb-b52b-a3597a9e521a-kube-api-access-nt649\") pod \"console-587867dc88-vzb79\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") " pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:07.987248 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:07.987226    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:08.129876 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:08.129842    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-587867dc88-vzb79"]
Jul 01 19:28:08.135585 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:28:08.135538    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f665472_c2bd_48fb_b52b_a3597a9e521a.slice/crio-94815079930c7720342335acfd3e8dd943e16b22f58078c41ee7275475ae0b8a WatchSource:0}: Error finding container 94815079930c7720342335acfd3e8dd943e16b22f58078c41ee7275475ae0b8a: Status 404 returned error can't find the container with id 94815079930c7720342335acfd3e8dd943e16b22f58078c41ee7275475ae0b8a
Jul 01 19:28:08.681251 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:08.681215    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55749f0c-bc29-49cb-9346-9687d3179626","Type":"ContainerStarted","Data":"34a822005d64e8a236c3e47a0c81861b1addbc4fee9de6bab65668168e6a3cd0"}
Jul 01 19:28:08.681693 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:08.681306    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55749f0c-bc29-49cb-9346-9687d3179626","Type":"ContainerStarted","Data":"a88beb66c70f4842ec69e665649c347e86bd2e07adcb8058eacce1d066770a48"}
Jul 01 19:28:08.681693 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:08.681346    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55749f0c-bc29-49cb-9346-9687d3179626","Type":"ContainerStarted","Data":"7d2d24789b4f7cf3878dfced799364a5befc898b2472178a8e5e210119a7e2de"}
Jul 01 19:28:08.681693 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:08.681376    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55749f0c-bc29-49cb-9346-9687d3179626","Type":"ContainerStarted","Data":"455ceaeef7609b009c2739550c00fa9ab1d3a4b2e8b3505bf3c05beae8c1e1be"}
Jul 01 19:28:08.681693 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:08.681400    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55749f0c-bc29-49cb-9346-9687d3179626","Type":"ContainerStarted","Data":"943aef3f0767eb2a62c4071853ca7d96ac2c2dc25c2b2279a80adbbdde832d00"}
Jul 01 19:28:08.681693 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:08.681439    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"55749f0c-bc29-49cb-9346-9687d3179626","Type":"ContainerStarted","Data":"94319000e19920cc5db08b598bbd952ed503cddadc2120258e38401e212533fc"}
Jul 01 19:28:08.682717 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:08.682693    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-587867dc88-vzb79" event={"ID":"6f665472-c2bd-48fb-b52b-a3597a9e521a","Type":"ContainerStarted","Data":"00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799"}
Jul 01 19:28:08.682792 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:08.682731    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-587867dc88-vzb79" event={"ID":"6f665472-c2bd-48fb-b52b-a3597a9e521a","Type":"ContainerStarted","Data":"94815079930c7720342335acfd3e8dd943e16b22f58078c41ee7275475ae0b8a"}
Jul 01 19:28:08.713043 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:08.712995    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.699455984 podStartE2EDuration="1.71298077s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.524786ms" isStatefulPod=false podCreationTimestamp="2026-07-01 19:28:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:28:08.710292906 +0000 UTC m=+399.945418251" watchObservedRunningTime="2026-07-01 19:28:08.71298077 +0000 UTC m=+399.948106081"
Jul 01 19:28:08.732716 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:08.732649    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-587867dc88-vzb79" podStartSLOduration=1.732626124 podStartE2EDuration="1.732626124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:28:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:28:08.730907584 +0000 UTC m=+399.966032921" watchObservedRunningTime="2026-07-01 19:28:08.732626124 +0000 UTC m=+399.967751438"
Jul 01 19:28:14.121355 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:28:14.121312    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zvhnr" podUID="c9d9a753-e701-4ae6-9c8a-41217d9a8f23"
Jul 01 19:28:14.121355 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:28:14.121323    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x" podUID="759856dd-e062-4e68-bbb5-9e17011f717e"
Jul 01 19:28:14.711076 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:14.711046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:28:14.711222 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:14.711051    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:28:16.175116 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:16.175076    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:28:16.175480 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:16.175127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:28:16.177648 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:16.177609    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c9d9a753-e701-4ae6-9c8a-41217d9a8f23-metrics-tls\") pod \"dns-default-zvhnr\" (UID: \"c9d9a753-e701-4ae6-9c8a-41217d9a8f23\") " pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:28:16.177756 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:16.177720    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/759856dd-e062-4e68-bbb5-9e17011f717e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-wc78x\" (UID: \"759856dd-e062-4e68-bbb5-9e17011f717e\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:28:16.214719 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:16.214690    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-b6cpl\""
Jul 01 19:28:16.215778 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:16.215763    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2g8hv\""
Jul 01 19:28:16.222076 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:16.222055    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:28:16.222128 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:16.222082    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"
Jul 01 19:28:16.379121 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:16.379095    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x"]
Jul 01 19:28:16.381832 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:28:16.381807    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod759856dd_e062_4e68_bbb5_9e17011f717e.slice/crio-31ea21793e7873895ab0467384118d7deb569d38ca9b2c60e34c7c5964dc7fa0 WatchSource:0}: Error finding container 31ea21793e7873895ab0467384118d7deb569d38ca9b2c60e34c7c5964dc7fa0: Status 404 returned error can't find the container with id 31ea21793e7873895ab0467384118d7deb569d38ca9b2c60e34c7c5964dc7fa0
Jul 01 19:28:16.401402 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:16.401379    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zvhnr"]
Jul 01 19:28:16.403593 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:28:16.403554    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9d9a753_e701_4ae6_9c8a_41217d9a8f23.slice/crio-c98e8b19ced7c63f54ac7c2eb3e104ff405d477ec6043ed3300c60ff0f43edc3 WatchSource:0}: Error finding container c98e8b19ced7c63f54ac7c2eb3e104ff405d477ec6043ed3300c60ff0f43edc3: Status 404 returned error can't find the container with id c98e8b19ced7c63f54ac7c2eb3e104ff405d477ec6043ed3300c60ff0f43edc3
Jul 01 19:28:16.720645 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:16.720613    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x" event={"ID":"759856dd-e062-4e68-bbb5-9e17011f717e","Type":"ContainerStarted","Data":"31ea21793e7873895ab0467384118d7deb569d38ca9b2c60e34c7c5964dc7fa0"}
Jul 01 19:28:16.721776 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:16.721748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvhnr" event={"ID":"c9d9a753-e701-4ae6-9c8a-41217d9a8f23","Type":"ContainerStarted","Data":"c98e8b19ced7c63f54ac7c2eb3e104ff405d477ec6043ed3300c60ff0f43edc3"}
Jul 01 19:28:17.987824 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:17.987791    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:17.988308 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:17.987954    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:17.993448 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:17.993423    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:18.731888 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:18.731848    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x" event={"ID":"759856dd-e062-4e68-bbb5-9e17011f717e","Type":"ContainerStarted","Data":"44aa420f54cd7f3f90125d86e4c992b6eda33786af3250e5f85f00db979b5f22"}
Jul 01 19:28:18.733718 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:18.733688    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvhnr" event={"ID":"c9d9a753-e701-4ae6-9c8a-41217d9a8f23","Type":"ContainerStarted","Data":"d6513f8f508f472437a947324be24c40d77e11c937facc04756ac64f59acbd77"}
Jul 01 19:28:18.733863 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:18.733738    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvhnr" event={"ID":"c9d9a753-e701-4ae6-9c8a-41217d9a8f23","Type":"ContainerStarted","Data":"b3e981c8ecd1f821e9c89e6895767f83b557bd7020ac212037349601a289fac2"}
Jul 01 19:28:18.733863 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:18.733842    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:28:18.737486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:18.737469    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:28:18.748645 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:18.748606    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-wc78x" podStartSLOduration=402.781005911 podStartE2EDuration="6m44.748592463s" totalImagesPullingTime="1.967586552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:28:18.748021869 +0000 UTC m=+409.983147182" watchObservedRunningTime="2026-07-01 19:28:18.748592463 +0000 UTC m=+409.983717767"
Jul 01 19:28:18.784382 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:18.784328    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zvhnr" podStartSLOduration=372.830230002 podStartE2EDuration="6m14.784311047s" totalImagesPullingTime="1.954081045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:28:18.783152069 +0000 UTC m=+410.018277405" watchObservedRunningTime="2026-07-01 19:28:18.784311047 +0000 UTC m=+410.019436359"
Jul 01 19:28:18.803338 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:18.803304    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dc9654d45-986nx"]
Jul 01 19:28:28.740928 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:28.740891    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zvhnr"
Jul 01 19:28:43.824993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:43.824944    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7dc9654d45-986nx" podUID="cb998592-faf1-4299-8bc4-0314e3b45329" containerName="console" containerID="cri-o://c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c" gracePeriod=15
Jul 01 19:28:44.069683 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.069656    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:28:44.220897 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.220872    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb998592-faf1-4299-8bc4-0314e3b45329-console-serving-cert\") pod \"cb998592-faf1-4299-8bc4-0314e3b45329\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") "
Jul 01 19:28:44.221063 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.220951    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-oauth-serving-cert\") pod \"cb998592-faf1-4299-8bc4-0314e3b45329\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") "
Jul 01 19:28:44.221063 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.220979    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8522s\" (UniqueName: \"kubernetes.io/projected/cb998592-faf1-4299-8bc4-0314e3b45329-kube-api-access-8522s\") pod \"cb998592-faf1-4299-8bc4-0314e3b45329\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") "
Jul 01 19:28:44.221063 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.221025    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb998592-faf1-4299-8bc4-0314e3b45329-console-oauth-config\") pod \"cb998592-faf1-4299-8bc4-0314e3b45329\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") "
Jul 01 19:28:44.221063 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.221048    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-console-config\") pod \"cb998592-faf1-4299-8bc4-0314e3b45329\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") "
Jul 01 19:28:44.221262 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.221073    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-trusted-ca-bundle\") pod \"cb998592-faf1-4299-8bc4-0314e3b45329\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") "
Jul 01 19:28:44.221262 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.221109    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-service-ca\") pod \"cb998592-faf1-4299-8bc4-0314e3b45329\" (UID: \"cb998592-faf1-4299-8bc4-0314e3b45329\") "
Jul 01 19:28:44.221380 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.221351    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cb998592-faf1-4299-8bc4-0314e3b45329" (UID: "cb998592-faf1-4299-8bc4-0314e3b45329"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:28:44.221488 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.221460    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-console-config" (OuterVolumeSpecName: "console-config") pod "cb998592-faf1-4299-8bc4-0314e3b45329" (UID: "cb998592-faf1-4299-8bc4-0314e3b45329"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:28:44.221617 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.221467    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cb998592-faf1-4299-8bc4-0314e3b45329" (UID: "cb998592-faf1-4299-8bc4-0314e3b45329"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:28:44.221743 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.221654    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-service-ca" (OuterVolumeSpecName: "service-ca") pod "cb998592-faf1-4299-8bc4-0314e3b45329" (UID: "cb998592-faf1-4299-8bc4-0314e3b45329"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:28:44.223181 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.223150    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb998592-faf1-4299-8bc4-0314e3b45329-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cb998592-faf1-4299-8bc4-0314e3b45329" (UID: "cb998592-faf1-4299-8bc4-0314e3b45329"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:28:44.223527 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.223506    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb998592-faf1-4299-8bc4-0314e3b45329-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cb998592-faf1-4299-8bc4-0314e3b45329" (UID: "cb998592-faf1-4299-8bc4-0314e3b45329"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:28:44.223527 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.223517    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb998592-faf1-4299-8bc4-0314e3b45329-kube-api-access-8522s" (OuterVolumeSpecName: "kube-api-access-8522s") pod "cb998592-faf1-4299-8bc4-0314e3b45329" (UID: "cb998592-faf1-4299-8bc4-0314e3b45329"). InnerVolumeSpecName "kube-api-access-8522s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:28:44.322438 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.322385    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-oauth-serving-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:44.322438 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.322432    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8522s\" (UniqueName: \"kubernetes.io/projected/cb998592-faf1-4299-8bc4-0314e3b45329-kube-api-access-8522s\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:44.322438 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.322447    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb998592-faf1-4299-8bc4-0314e3b45329-console-oauth-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:44.322715 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.322462    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-console-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:44.322715 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.322473    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-trusted-ca-bundle\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:44.322715 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.322486    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb998592-faf1-4299-8bc4-0314e3b45329-service-ca\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:44.322715 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.322498    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb998592-faf1-4299-8bc4-0314e3b45329-console-serving-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:28:44.845653 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.845620    2581 generic.go:356] "Generic (PLEG): container finished" podID="cb998592-faf1-4299-8bc4-0314e3b45329" containerID="c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c" exitCode=0
Jul 01 19:28:44.846115 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.845667    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc9654d45-986nx"
Jul 01 19:28:44.846115 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.845710    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc9654d45-986nx" event={"ID":"cb998592-faf1-4299-8bc4-0314e3b45329","Type":"ContainerDied","Data":"c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c"}
Jul 01 19:28:44.846115 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.845774    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc9654d45-986nx" event={"ID":"cb998592-faf1-4299-8bc4-0314e3b45329","Type":"ContainerDied","Data":"599c589df4df31cd512c51b91a27641509d1d2a0b8bc37baa3b13d9949084a4b"}
Jul 01 19:28:44.846115 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.845807    2581 scope.go:117] "RemoveContainer" containerID="c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c"
Jul 01 19:28:44.855156 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.855139    2581 scope.go:117] "RemoveContainer" containerID="c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c"
Jul 01 19:28:44.855393 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:28:44.855375    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c\": container with ID starting with c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c not found: ID does not exist" containerID="c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c"
Jul 01 19:28:44.855450 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.855402    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c"} err="failed to get container status \"c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c\": rpc error: code = NotFound desc = could not find container \"c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c\": container with ID starting with c5426b33c8e3147921df2ed70837a027bae2af8e78d1e00e491d41567510f49c not found: ID does not exist"
Jul 01 19:28:44.868088 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.868063    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dc9654d45-986nx"]
Jul 01 19:28:44.871906 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:44.871887    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dc9654d45-986nx"]
Jul 01 19:28:45.214863 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:28:45.214830    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb998592-faf1-4299-8bc4-0314e3b45329" path="/var/lib/kubelet/pods/cb998592-faf1-4299-8bc4-0314e3b45329/volumes"
Jul 01 19:29:18.386226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.386188    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"]
Jul 01 19:29:18.386772 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.386541    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb998592-faf1-4299-8bc4-0314e3b45329" containerName="console"
Jul 01 19:29:18.386772 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.386552    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb998592-faf1-4299-8bc4-0314e3b45329" containerName="console"
Jul 01 19:29:18.386772 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.386625    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb998592-faf1-4299-8bc4-0314e3b45329" containerName="console"
Jul 01 19:29:18.389827 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.389811    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"
Jul 01 19:29:18.393353 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.393328    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 01 19:29:18.393484 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.393328    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 01 19:29:18.393484 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.393427    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-5cz9n\""
Jul 01 19:29:18.393484 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.393364    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 01 19:29:18.404506 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.404484    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"]
Jul 01 19:29:18.505249 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.505210    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcjzh\" (UniqueName: \"kubernetes.io/projected/e9408e93-4d7d-43f9-8014-282d04b2befd-kube-api-access-rcjzh\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-l28hb\" (UID: \"e9408e93-4d7d-43f9-8014-282d04b2befd\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"
Jul 01 19:29:18.505433 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.505256    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/e9408e93-4d7d-43f9-8014-282d04b2befd-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-l28hb\" (UID: \"e9408e93-4d7d-43f9-8014-282d04b2befd\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"
Jul 01 19:29:18.605919 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.605881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcjzh\" (UniqueName: \"kubernetes.io/projected/e9408e93-4d7d-43f9-8014-282d04b2befd-kube-api-access-rcjzh\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-l28hb\" (UID: \"e9408e93-4d7d-43f9-8014-282d04b2befd\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"
Jul 01 19:29:18.605919 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.605925    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/e9408e93-4d7d-43f9-8014-282d04b2befd-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-l28hb\" (UID: \"e9408e93-4d7d-43f9-8014-282d04b2befd\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"
Jul 01 19:29:18.608309 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.608283    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/e9408e93-4d7d-43f9-8014-282d04b2befd-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-l28hb\" (UID: \"e9408e93-4d7d-43f9-8014-282d04b2befd\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"
Jul 01 19:29:18.618491 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.618454    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcjzh\" (UniqueName: \"kubernetes.io/projected/e9408e93-4d7d-43f9-8014-282d04b2befd-kube-api-access-rcjzh\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-l28hb\" (UID: \"e9408e93-4d7d-43f9-8014-282d04b2befd\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"
Jul 01 19:29:18.699613 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.699497    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"
Jul 01 19:29:18.848074 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.848051    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"]
Jul 01 19:29:18.850176 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:29:18.850149    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9408e93_4d7d_43f9_8014_282d04b2befd.slice/crio-8fb525672415d45835d6924f872ae20bccf1b74d9c8a7898d73e1e9b0008f72f WatchSource:0}: Error finding container 8fb525672415d45835d6924f872ae20bccf1b74d9c8a7898d73e1e9b0008f72f: Status 404 returned error can't find the container with id 8fb525672415d45835d6924f872ae20bccf1b74d9c8a7898d73e1e9b0008f72f
Jul 01 19:29:18.991689 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:18.991662    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb" event={"ID":"e9408e93-4d7d-43f9-8014-282d04b2befd","Type":"ContainerStarted","Data":"8fb525672415d45835d6924f872ae20bccf1b74d9c8a7898d73e1e9b0008f72f"}
Jul 01 19:29:23.017843 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:23.017804    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb" event={"ID":"e9408e93-4d7d-43f9-8014-282d04b2befd","Type":"ContainerStarted","Data":"2257d294bf5658fa5b56f233dd19d9177d4b1dc011f9efe0c0853521ff8a838a"}
Jul 01 19:29:23.018222 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:23.017925    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"
Jul 01 19:29:23.043882 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:23.043832    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb" podStartSLOduration=1.7874818399999999 podStartE2EDuration="5.043816221s" totalImagesPullingTime="3.256334381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:29:23.041825237 +0000 UTC m=+474.276950551" watchObservedRunningTime="2026-07-01 19:29:23.043816221 +0000 UTC m=+474.278941532"
Jul 01 19:29:44.025473 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:29:44.025440    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-l28hb"
Jul 01 19:30:31.341329 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.341250    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-bd65f9df8-d5ntn"]
Jul 01 19:30:31.343638 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.343621    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn"
Jul 01 19:30:31.347235 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.347214    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-qvxwb\""
Jul 01 19:30:31.347235 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.347227    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 01 19:30:31.348195 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.348178    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 01 19:30:31.348270 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.348235    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 01 19:30:31.362802 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.362772    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-bd65f9df8-d5ntn"]
Jul 01 19:30:31.384107 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.384071    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-stp9g"]
Jul 01 19:30:31.386501 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.386462    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g"
Jul 01 19:30:31.390846 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.390822    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-6x72p\""
Jul 01 19:30:31.391774 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.391757    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 01 19:30:31.411286 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.411256    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-stp9g"]
Jul 01 19:30:31.433547 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.433516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9dsn\" (UniqueName: \"kubernetes.io/projected/5274c797-5920-4430-9fd1-2d0996055b08-kube-api-access-f9dsn\") pod \"kserve-controller-manager-bd65f9df8-d5ntn\" (UID: \"5274c797-5920-4430-9fd1-2d0996055b08\") " pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn"
Jul 01 19:30:31.433731 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.433617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5274c797-5920-4430-9fd1-2d0996055b08-cert\") pod \"kserve-controller-manager-bd65f9df8-d5ntn\" (UID: \"5274c797-5920-4430-9fd1-2d0996055b08\") " pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn"
Jul 01 19:30:31.534642 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.534571    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f221aef-6c2e-4e56-9903-99cde5bf0f98-cert\") pod \"llmisvc-controller-manager-67d95fdc48-stp9g\" (UID: \"2f221aef-6c2e-4e56-9903-99cde5bf0f98\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g"
Jul 01 19:30:31.534642 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.534652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5274c797-5920-4430-9fd1-2d0996055b08-cert\") pod \"kserve-controller-manager-bd65f9df8-d5ntn\" (UID: \"5274c797-5920-4430-9fd1-2d0996055b08\") " pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn"
Jul 01 19:30:31.534921 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.534682    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmdn6\" (UniqueName: \"kubernetes.io/projected/2f221aef-6c2e-4e56-9903-99cde5bf0f98-kube-api-access-vmdn6\") pod \"llmisvc-controller-manager-67d95fdc48-stp9g\" (UID: \"2f221aef-6c2e-4e56-9903-99cde5bf0f98\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g"
Jul 01 19:30:31.534921 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.534741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9dsn\" (UniqueName: \"kubernetes.io/projected/5274c797-5920-4430-9fd1-2d0996055b08-kube-api-access-f9dsn\") pod \"kserve-controller-manager-bd65f9df8-d5ntn\" (UID: \"5274c797-5920-4430-9fd1-2d0996055b08\") " pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn"
Jul 01 19:30:31.537075 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.537052    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5274c797-5920-4430-9fd1-2d0996055b08-cert\") pod \"kserve-controller-manager-bd65f9df8-d5ntn\" (UID: \"5274c797-5920-4430-9fd1-2d0996055b08\") " pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn"
Jul 01 19:30:31.550140 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.550110    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9dsn\" (UniqueName: \"kubernetes.io/projected/5274c797-5920-4430-9fd1-2d0996055b08-kube-api-access-f9dsn\") pod \"kserve-controller-manager-bd65f9df8-d5ntn\" (UID: \"5274c797-5920-4430-9fd1-2d0996055b08\") " pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn"
Jul 01 19:30:31.635734 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.635627    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f221aef-6c2e-4e56-9903-99cde5bf0f98-cert\") pod \"llmisvc-controller-manager-67d95fdc48-stp9g\" (UID: \"2f221aef-6c2e-4e56-9903-99cde5bf0f98\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g"
Jul 01 19:30:31.635734 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.635696    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdn6\" (UniqueName: \"kubernetes.io/projected/2f221aef-6c2e-4e56-9903-99cde5bf0f98-kube-api-access-vmdn6\") pod \"llmisvc-controller-manager-67d95fdc48-stp9g\" (UID: \"2f221aef-6c2e-4e56-9903-99cde5bf0f98\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g"
Jul 01 19:30:31.635923 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:30:31.635790    2581 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 01 19:30:31.635923 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:30:31.635852    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f221aef-6c2e-4e56-9903-99cde5bf0f98-cert podName:2f221aef-6c2e-4e56-9903-99cde5bf0f98 nodeName:}" failed. No retries permitted until 2026-07-01 19:30:32.135835688 +0000 UTC m=+543.370960991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f221aef-6c2e-4e56-9903-99cde5bf0f98-cert") pod "llmisvc-controller-manager-67d95fdc48-stp9g" (UID: "2f221aef-6c2e-4e56-9903-99cde5bf0f98") : secret "llmisvc-webhook-server-cert" not found
Jul 01 19:30:31.649648 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.649616    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmdn6\" (UniqueName: \"kubernetes.io/projected/2f221aef-6c2e-4e56-9903-99cde5bf0f98-kube-api-access-vmdn6\") pod \"llmisvc-controller-manager-67d95fdc48-stp9g\" (UID: \"2f221aef-6c2e-4e56-9903-99cde5bf0f98\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g"
Jul 01 19:30:31.653456 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.653434    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn"
Jul 01 19:30:31.843294 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:31.843251    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-bd65f9df8-d5ntn"]
Jul 01 19:30:31.846267 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:30:31.846242    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5274c797_5920_4430_9fd1_2d0996055b08.slice/crio-011573e087a7277fcc2e23574573c7ab7d66fe58cb979ce9a7a1c8ae1f68c09d WatchSource:0}: Error finding container 011573e087a7277fcc2e23574573c7ab7d66fe58cb979ce9a7a1c8ae1f68c09d: Status 404 returned error can't find the container with id 011573e087a7277fcc2e23574573c7ab7d66fe58cb979ce9a7a1c8ae1f68c09d
Jul 01 19:30:32.141078 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:32.141046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f221aef-6c2e-4e56-9903-99cde5bf0f98-cert\") pod \"llmisvc-controller-manager-67d95fdc48-stp9g\" (UID: \"2f221aef-6c2e-4e56-9903-99cde5bf0f98\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g"
Jul 01 19:30:32.143408 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:32.143384    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f221aef-6c2e-4e56-9903-99cde5bf0f98-cert\") pod \"llmisvc-controller-manager-67d95fdc48-stp9g\" (UID: \"2f221aef-6c2e-4e56-9903-99cde5bf0f98\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g"
Jul 01 19:30:32.296722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:32.296690    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g"
Jul 01 19:30:32.332084 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:32.332048    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn" event={"ID":"5274c797-5920-4430-9fd1-2d0996055b08","Type":"ContainerStarted","Data":"011573e087a7277fcc2e23574573c7ab7d66fe58cb979ce9a7a1c8ae1f68c09d"}
Jul 01 19:30:32.436076 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:32.435993    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-stp9g"]
Jul 01 19:30:32.439061 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:30:32.439033    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2f221aef_6c2e_4e56_9903_99cde5bf0f98.slice/crio-6a16402415e70caa06b5f6e46d4453304c4d81ad1300e51887645130cef945dc WatchSource:0}: Error finding container 6a16402415e70caa06b5f6e46d4453304c4d81ad1300e51887645130cef945dc: Status 404 returned error can't find the container with id 6a16402415e70caa06b5f6e46d4453304c4d81ad1300e51887645130cef945dc
Jul 01 19:30:33.340724 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:33.340687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g" event={"ID":"2f221aef-6c2e-4e56-9903-99cde5bf0f98","Type":"ContainerStarted","Data":"6a16402415e70caa06b5f6e46d4453304c4d81ad1300e51887645130cef945dc"}
Jul 01 19:30:36.357785 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:36.357748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g" event={"ID":"2f221aef-6c2e-4e56-9903-99cde5bf0f98","Type":"ContainerStarted","Data":"9c3066b08449d0b45c98e5124d93bf39235b417b11053fe36eeb8265e2ff0300"}
Jul 01 19:30:36.358263 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:36.357866    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g"
Jul 01 19:30:36.359282 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:36.359259    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn" event={"ID":"5274c797-5920-4430-9fd1-2d0996055b08","Type":"ContainerStarted","Data":"4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773"}
Jul 01 19:30:36.359404 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:36.359385    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn"
Jul 01 19:30:36.382206 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:36.382157    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g" podStartSLOduration=2.303809074 podStartE2EDuration="5.382141786s" totalImagesPullingTime="3.078332712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:30:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:30:36.380349179 +0000 UTC m=+547.615474493" watchObservedRunningTime="2026-07-01 19:30:36.382141786 +0000 UTC m=+547.617267097"
Jul 01 19:30:36.402340 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:30:36.402288    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn" podStartSLOduration=1.7843001969999999 podStartE2EDuration="5.402271858s" totalImagesPullingTime="3.617971661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:30:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:30:36.400542783 +0000 UTC m=+547.635668092" watchObservedRunningTime="2026-07-01 19:30:36.402271858 +0000 UTC m=+547.637397170"
Jul 01 19:31:07.366293 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:07.366265    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-stp9g"
Jul 01 19:31:07.369135 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:07.369120    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn"
Jul 01 19:31:08.741231 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:08.741195    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-bd65f9df8-d5ntn"]
Jul 01 19:31:08.741631 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:08.741361    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn" podUID="5274c797-5920-4430-9fd1-2d0996055b08" containerName="manager" containerID="cri-o://4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773" gracePeriod=10
Jul 01 19:31:08.766386 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:08.766353    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-bd65f9df8-llt7t"]
Jul 01 19:31:08.816528 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:08.816503    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-bd65f9df8-llt7t"]
Jul 01 19:31:08.816640 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:08.816622    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-bd65f9df8-llt7t"
Jul 01 19:31:08.853646 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:08.853610    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9844155-0df5-4ae6-8a2d-628ab0769a2e-cert\") pod \"kserve-controller-manager-bd65f9df8-llt7t\" (UID: \"b9844155-0df5-4ae6-8a2d-628ab0769a2e\") " pod="kserve/kserve-controller-manager-bd65f9df8-llt7t"
Jul 01 19:31:08.853814 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:08.853652    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gvnf\" (UniqueName: \"kubernetes.io/projected/b9844155-0df5-4ae6-8a2d-628ab0769a2e-kube-api-access-4gvnf\") pod \"kserve-controller-manager-bd65f9df8-llt7t\" (UID: \"b9844155-0df5-4ae6-8a2d-628ab0769a2e\") " pod="kserve/kserve-controller-manager-bd65f9df8-llt7t"
Jul 01 19:31:08.955148 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:08.955116    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gvnf\" (UniqueName: \"kubernetes.io/projected/b9844155-0df5-4ae6-8a2d-628ab0769a2e-kube-api-access-4gvnf\") pod \"kserve-controller-manager-bd65f9df8-llt7t\" (UID: \"b9844155-0df5-4ae6-8a2d-628ab0769a2e\") " pod="kserve/kserve-controller-manager-bd65f9df8-llt7t"
Jul 01 19:31:08.955302 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:08.955246    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9844155-0df5-4ae6-8a2d-628ab0769a2e-cert\") pod \"kserve-controller-manager-bd65f9df8-llt7t\" (UID: \"b9844155-0df5-4ae6-8a2d-628ab0769a2e\") " pod="kserve/kserve-controller-manager-bd65f9df8-llt7t"
Jul 01 19:31:08.957624 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:08.957602    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9844155-0df5-4ae6-8a2d-628ab0769a2e-cert\") pod \"kserve-controller-manager-bd65f9df8-llt7t\" (UID: \"b9844155-0df5-4ae6-8a2d-628ab0769a2e\") " pod="kserve/kserve-controller-manager-bd65f9df8-llt7t"
Jul 01 19:31:08.964454 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:08.964426    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gvnf\" (UniqueName: \"kubernetes.io/projected/b9844155-0df5-4ae6-8a2d-628ab0769a2e-kube-api-access-4gvnf\") pod \"kserve-controller-manager-bd65f9df8-llt7t\" (UID: \"b9844155-0df5-4ae6-8a2d-628ab0769a2e\") " pod="kserve/kserve-controller-manager-bd65f9df8-llt7t"
Jul 01 19:31:09.027822 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.027791    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn"
Jul 01 19:31:09.156882 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.156848    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9dsn\" (UniqueName: \"kubernetes.io/projected/5274c797-5920-4430-9fd1-2d0996055b08-kube-api-access-f9dsn\") pod \"5274c797-5920-4430-9fd1-2d0996055b08\" (UID: \"5274c797-5920-4430-9fd1-2d0996055b08\") "
Jul 01 19:31:09.156882 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.156885    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5274c797-5920-4430-9fd1-2d0996055b08-cert\") pod \"5274c797-5920-4430-9fd1-2d0996055b08\" (UID: \"5274c797-5920-4430-9fd1-2d0996055b08\") "
Jul 01 19:31:09.159069 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.159043    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5274c797-5920-4430-9fd1-2d0996055b08-cert" (OuterVolumeSpecName: "cert") pod "5274c797-5920-4430-9fd1-2d0996055b08" (UID: "5274c797-5920-4430-9fd1-2d0996055b08"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:31:09.159177 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.159043    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5274c797-5920-4430-9fd1-2d0996055b08-kube-api-access-f9dsn" (OuterVolumeSpecName: "kube-api-access-f9dsn") pod "5274c797-5920-4430-9fd1-2d0996055b08" (UID: "5274c797-5920-4430-9fd1-2d0996055b08"). InnerVolumeSpecName "kube-api-access-f9dsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:31:09.189229 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.189201    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-bd65f9df8-llt7t"
Jul 01 19:31:09.257634 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.257504    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9dsn\" (UniqueName: \"kubernetes.io/projected/5274c797-5920-4430-9fd1-2d0996055b08-kube-api-access-f9dsn\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:31:09.257781 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.257650    2581 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5274c797-5920-4430-9fd1-2d0996055b08-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:31:09.348357 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.348333    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-bd65f9df8-llt7t"]
Jul 01 19:31:09.349819 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:31:09.349795    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9844155_0df5_4ae6_8a2d_628ab0769a2e.slice/crio-5d343c58583081a2ff65218ec683ef313fa89dfe9ef16b12941f64bb8f9ce6a5 WatchSource:0}: Error finding container 5d343c58583081a2ff65218ec683ef313fa89dfe9ef16b12941f64bb8f9ce6a5: Status 404 returned error can't find the container with id 5d343c58583081a2ff65218ec683ef313fa89dfe9ef16b12941f64bb8f9ce6a5
Jul 01 19:31:09.510555 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.510432    2581 generic.go:356] "Generic (PLEG): container finished" podID="5274c797-5920-4430-9fd1-2d0996055b08" containerID="4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773" exitCode=0
Jul 01 19:31:09.510555 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.510529    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn"
Jul 01 19:31:09.510555 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.510530    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn" event={"ID":"5274c797-5920-4430-9fd1-2d0996055b08","Type":"ContainerDied","Data":"4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773"}
Jul 01 19:31:09.510858 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.510615    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-bd65f9df8-d5ntn" event={"ID":"5274c797-5920-4430-9fd1-2d0996055b08","Type":"ContainerDied","Data":"011573e087a7277fcc2e23574573c7ab7d66fe58cb979ce9a7a1c8ae1f68c09d"}
Jul 01 19:31:09.510858 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.510647    2581 scope.go:117] "RemoveContainer" containerID="4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773"
Jul 01 19:31:09.511932 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.511914    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-bd65f9df8-llt7t" event={"ID":"b9844155-0df5-4ae6-8a2d-628ab0769a2e","Type":"ContainerStarted","Data":"5d343c58583081a2ff65218ec683ef313fa89dfe9ef16b12941f64bb8f9ce6a5"}
Jul 01 19:31:09.518854 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.518838    2581 scope.go:117] "RemoveContainer" containerID="4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773"
Jul 01 19:31:09.519113 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:31:09.519096    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773\": container with ID starting with 4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773 not found: ID does not exist" containerID="4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773"
Jul 01 19:31:09.519171 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.519120    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773"} err="failed to get container status \"4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773\": rpc error: code = NotFound desc = could not find container \"4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773\": container with ID starting with 4c6c2748fe7a0a04dbe6ceff1ebd2ace38c7f120c1a96874fead3e5928484773 not found: ID does not exist"
Jul 01 19:31:09.529896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.529878    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-bd65f9df8-d5ntn"]
Jul 01 19:31:09.535045 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:09.535026    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-bd65f9df8-d5ntn"]
Jul 01 19:31:10.518755 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:10.518717    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-bd65f9df8-llt7t" event={"ID":"b9844155-0df5-4ae6-8a2d-628ab0769a2e","Type":"ContainerStarted","Data":"c8f436759455cfbd4bd661a1d2f37bb7021cfdbe7584fac53adcf4d4b9dcb56c"}
Jul 01 19:31:10.519184 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:10.518795    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-bd65f9df8-llt7t"
Jul 01 19:31:10.537768 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:10.537721    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-bd65f9df8-llt7t" podStartSLOduration=1.974169314 podStartE2EDuration="2.537706117s" totalImagesPullingTime="563.536803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:31:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:31:10.535952912 +0000 UTC m=+581.771078225" watchObservedRunningTime="2026-07-01 19:31:10.537706117 +0000 UTC m=+581.772831480"
Jul 01 19:31:11.215758 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:11.215729    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5274c797-5920-4430-9fd1-2d0996055b08" path="/var/lib/kubelet/pods/5274c797-5920-4430-9fd1-2d0996055b08/volumes"
Jul 01 19:31:29.173387 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:29.173361    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:31:29.173851 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:29.173361    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:31:41.527722 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:41.527689    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-bd65f9df8-llt7t"
Jul 01 19:31:42.542501 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.542459    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-dkrvk"]
Jul 01 19:31:42.542876 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.542826    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5274c797-5920-4430-9fd1-2d0996055b08" containerName="manager"
Jul 01 19:31:42.542876 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.542838    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5274c797-5920-4430-9fd1-2d0996055b08" containerName="manager"
Jul 01 19:31:42.542952 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.542905    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5274c797-5920-4430-9fd1-2d0996055b08" containerName="manager"
Jul 01 19:31:42.545735 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.545717    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-dkrvk"
Jul 01 19:31:42.548865 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.548843    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jul 01 19:31:42.549060 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.549046    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-zb7ng\""
Jul 01 19:31:42.555867 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.555850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49771f33-bfba-4947-b06c-b1cd7de176f6-cert\") pod \"odh-model-controller-84b54b7f97-dkrvk\" (UID: \"49771f33-bfba-4947-b06c-b1cd7de176f6\") " pod="kserve/odh-model-controller-84b54b7f97-dkrvk"
Jul 01 19:31:42.555933 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.555913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89g4h\" (UniqueName: \"kubernetes.io/projected/49771f33-bfba-4947-b06c-b1cd7de176f6-kube-api-access-89g4h\") pod \"odh-model-controller-84b54b7f97-dkrvk\" (UID: \"49771f33-bfba-4947-b06c-b1cd7de176f6\") " pod="kserve/odh-model-controller-84b54b7f97-dkrvk"
Jul 01 19:31:42.561878 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.561819    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-dkrvk"]
Jul 01 19:31:42.657058 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.657022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49771f33-bfba-4947-b06c-b1cd7de176f6-cert\") pod \"odh-model-controller-84b54b7f97-dkrvk\" (UID: \"49771f33-bfba-4947-b06c-b1cd7de176f6\") " pod="kserve/odh-model-controller-84b54b7f97-dkrvk"
Jul 01 19:31:42.657239 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.657111    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89g4h\" (UniqueName: \"kubernetes.io/projected/49771f33-bfba-4947-b06c-b1cd7de176f6-kube-api-access-89g4h\") pod \"odh-model-controller-84b54b7f97-dkrvk\" (UID: \"49771f33-bfba-4947-b06c-b1cd7de176f6\") " pod="kserve/odh-model-controller-84b54b7f97-dkrvk"
Jul 01 19:31:42.659533 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.659508    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49771f33-bfba-4947-b06c-b1cd7de176f6-cert\") pod \"odh-model-controller-84b54b7f97-dkrvk\" (UID: \"49771f33-bfba-4947-b06c-b1cd7de176f6\") " pod="kserve/odh-model-controller-84b54b7f97-dkrvk"
Jul 01 19:31:42.665858 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.665833    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89g4h\" (UniqueName: \"kubernetes.io/projected/49771f33-bfba-4947-b06c-b1cd7de176f6-kube-api-access-89g4h\") pod \"odh-model-controller-84b54b7f97-dkrvk\" (UID: \"49771f33-bfba-4947-b06c-b1cd7de176f6\") " pod="kserve/odh-model-controller-84b54b7f97-dkrvk"
Jul 01 19:31:42.856226 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.856132    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-dkrvk"
Jul 01 19:31:42.987915 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.987602    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-dkrvk"]
Jul 01 19:31:42.989755 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:31:42.989727    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49771f33_bfba_4947_b06c_b1cd7de176f6.slice/crio-ed54d7323474bd2b0b7f8d71bd81826749bd361f7ed2babc594df2afb9c5c87f WatchSource:0}: Error finding container ed54d7323474bd2b0b7f8d71bd81826749bd361f7ed2babc594df2afb9c5c87f: Status 404 returned error can't find the container with id ed54d7323474bd2b0b7f8d71bd81826749bd361f7ed2babc594df2afb9c5c87f
Jul 01 19:31:42.991050 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:42.991033    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 19:31:43.671084 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:43.671045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-dkrvk" event={"ID":"49771f33-bfba-4947-b06c-b1cd7de176f6","Type":"ContainerStarted","Data":"ed54d7323474bd2b0b7f8d71bd81826749bd361f7ed2babc594df2afb9c5c87f"}
Jul 01 19:31:46.690220 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:46.690180    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-dkrvk" event={"ID":"49771f33-bfba-4947-b06c-b1cd7de176f6","Type":"ContainerStarted","Data":"76703c61ac3d3b9141296a880511caf52453a7dddafd7363e195debe782b509d"}
Jul 01 19:31:46.690622 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:46.690370    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-dkrvk"
Jul 01 19:31:46.709050 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:46.708996    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-dkrvk" podStartSLOduration=1.829788271 podStartE2EDuration="4.708976673s" totalImagesPullingTime="2.879188402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:31:46.707504762 +0000 UTC m=+617.942630075" watchObservedRunningTime="2026-07-01 19:31:46.708976673 +0000 UTC m=+617.944101988"
Jul 01 19:31:57.699394 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:31:57.699361    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-dkrvk"
Jul 01 19:32:28.198623 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.198589    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b97df4676-2f4lw"]
Jul 01 19:32:28.201289 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.201266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.214322 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.214292    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b97df4676-2f4lw"]
Jul 01 19:32:28.345716 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.345667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3f025d-1666-47a8-8b45-94fd15cce2e3-service-ca\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.346004 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.345762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f3f025d-1666-47a8-8b45-94fd15cce2e3-console-config\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.346004 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.345795    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b52nq\" (UniqueName: \"kubernetes.io/projected/8f3f025d-1666-47a8-8b45-94fd15cce2e3-kube-api-access-b52nq\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.346004 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.345853    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f3f025d-1666-47a8-8b45-94fd15cce2e3-console-serving-cert\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.346004 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.345890    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f3f025d-1666-47a8-8b45-94fd15cce2e3-console-oauth-config\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.346004 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.345923    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f3f025d-1666-47a8-8b45-94fd15cce2e3-oauth-serving-cert\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.346004 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.345955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f3f025d-1666-47a8-8b45-94fd15cce2e3-trusted-ca-bundle\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.447019 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.446981    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f3f025d-1666-47a8-8b45-94fd15cce2e3-trusted-ca-bundle\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.447019 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.447028    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3f025d-1666-47a8-8b45-94fd15cce2e3-service-ca\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.447239 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.447065    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f3f025d-1666-47a8-8b45-94fd15cce2e3-console-config\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.447277 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.447249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b52nq\" (UniqueName: \"kubernetes.io/projected/8f3f025d-1666-47a8-8b45-94fd15cce2e3-kube-api-access-b52nq\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.447316 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.447301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f3f025d-1666-47a8-8b45-94fd15cce2e3-console-serving-cert\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.447349 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.447326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f3f025d-1666-47a8-8b45-94fd15cce2e3-console-oauth-config\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.447382 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.447366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f3f025d-1666-47a8-8b45-94fd15cce2e3-oauth-serving-cert\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.447808 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.447781    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f3f025d-1666-47a8-8b45-94fd15cce2e3-console-config\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.447952 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.447869    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3f025d-1666-47a8-8b45-94fd15cce2e3-service-ca\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.448027 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.447953    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f3f025d-1666-47a8-8b45-94fd15cce2e3-trusted-ca-bundle\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.448093 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.448074    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f3f025d-1666-47a8-8b45-94fd15cce2e3-oauth-serving-cert\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.449831 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.449780    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f3f025d-1666-47a8-8b45-94fd15cce2e3-console-serving-cert\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.449927 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.449837    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f3f025d-1666-47a8-8b45-94fd15cce2e3-console-oauth-config\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.457530 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.457511    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b52nq\" (UniqueName: \"kubernetes.io/projected/8f3f025d-1666-47a8-8b45-94fd15cce2e3-kube-api-access-b52nq\") pod \"console-5b97df4676-2f4lw\" (UID: \"8f3f025d-1666-47a8-8b45-94fd15cce2e3\") " pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.510417 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.510382    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:28.650115 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.650087    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b97df4676-2f4lw"]
Jul 01 19:32:28.651775 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:32:28.651749    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f3f025d_1666_47a8_8b45_94fd15cce2e3.slice/crio-20ae6f26ac168b04df5d6c591cdba0f03259f67e8fef53b03f6f1a4f30c4303a WatchSource:0}: Error finding container 20ae6f26ac168b04df5d6c591cdba0f03259f67e8fef53b03f6f1a4f30c4303a: Status 404 returned error can't find the container with id 20ae6f26ac168b04df5d6c591cdba0f03259f67e8fef53b03f6f1a4f30c4303a
Jul 01 19:32:28.890837 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.890799    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b97df4676-2f4lw" event={"ID":"8f3f025d-1666-47a8-8b45-94fd15cce2e3","Type":"ContainerStarted","Data":"f3eac5b5a5c0dbc1a4bda2b2d5085359ce2173d7e16e0373b9920cba637fd86b"}
Jul 01 19:32:28.891011 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.890854    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b97df4676-2f4lw" event={"ID":"8f3f025d-1666-47a8-8b45-94fd15cce2e3","Type":"ContainerStarted","Data":"20ae6f26ac168b04df5d6c591cdba0f03259f67e8fef53b03f6f1a4f30c4303a"}
Jul 01 19:32:28.910394 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:28.910343    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b97df4676-2f4lw" podStartSLOduration=0.910328093 podStartE2EDuration="910.328093ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:32:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:32:28.909394529 +0000 UTC m=+660.144519884" watchObservedRunningTime="2026-07-01 19:32:28.910328093 +0000 UTC m=+660.145453404"
Jul 01 19:32:38.511344 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:38.511305    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:38.511756 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:38.511371    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:38.516198 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:38.516176    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:38.940389 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:38.940312    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b97df4676-2f4lw"
Jul 01 19:32:38.997461 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:32:38.997430    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-587867dc88-vzb79"]
Jul 01 19:33:04.026103 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.026041    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-587867dc88-vzb79" podUID="6f665472-c2bd-48fb-b52b-a3597a9e521a" containerName="console" containerID="cri-o://00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799" gracePeriod=15
Jul 01 19:33:04.282384 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.282324    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:33:04.351183 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.351148    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-trusted-ca-bundle\") pod \"6f665472-c2bd-48fb-b52b-a3597a9e521a\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") "
Jul 01 19:33:04.351183 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.351185    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-oauth-serving-cert\") pod \"6f665472-c2bd-48fb-b52b-a3597a9e521a\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") "
Jul 01 19:33:04.351414 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.351247    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nt649\" (UniqueName: \"kubernetes.io/projected/6f665472-c2bd-48fb-b52b-a3597a9e521a-kube-api-access-nt649\") pod \"6f665472-c2bd-48fb-b52b-a3597a9e521a\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") "
Jul 01 19:33:04.351414 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.351272    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-oauth-config\") pod \"6f665472-c2bd-48fb-b52b-a3597a9e521a\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") "
Jul 01 19:33:04.351414 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.351291    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-service-ca\") pod \"6f665472-c2bd-48fb-b52b-a3597a9e521a\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") "
Jul 01 19:33:04.351414 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.351385    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-config\") pod \"6f665472-c2bd-48fb-b52b-a3597a9e521a\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") "
Jul 01 19:33:04.351623 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.351489    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-serving-cert\") pod \"6f665472-c2bd-48fb-b52b-a3597a9e521a\" (UID: \"6f665472-c2bd-48fb-b52b-a3597a9e521a\") "
Jul 01 19:33:04.351859 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.351653    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6f665472-c2bd-48fb-b52b-a3597a9e521a" (UID: "6f665472-c2bd-48fb-b52b-a3597a9e521a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:33:04.351859 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.351780    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6f665472-c2bd-48fb-b52b-a3597a9e521a" (UID: "6f665472-c2bd-48fb-b52b-a3597a9e521a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:33:04.351859 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.351790    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-service-ca" (OuterVolumeSpecName: "service-ca") pod "6f665472-c2bd-48fb-b52b-a3597a9e521a" (UID: "6f665472-c2bd-48fb-b52b-a3597a9e521a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:33:04.351859 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.351849    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-config" (OuterVolumeSpecName: "console-config") pod "6f665472-c2bd-48fb-b52b-a3597a9e521a" (UID: "6f665472-c2bd-48fb-b52b-a3597a9e521a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:33:04.353625 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.353599    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6f665472-c2bd-48fb-b52b-a3597a9e521a" (UID: "6f665472-c2bd-48fb-b52b-a3597a9e521a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:33:04.353736 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.353642    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f665472-c2bd-48fb-b52b-a3597a9e521a-kube-api-access-nt649" (OuterVolumeSpecName: "kube-api-access-nt649") pod "6f665472-c2bd-48fb-b52b-a3597a9e521a" (UID: "6f665472-c2bd-48fb-b52b-a3597a9e521a"). InnerVolumeSpecName "kube-api-access-nt649". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:33:04.353736 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.353696    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6f665472-c2bd-48fb-b52b-a3597a9e521a" (UID: "6f665472-c2bd-48fb-b52b-a3597a9e521a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:33:04.452767 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.452733    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-trusted-ca-bundle\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:33:04.452767 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.452759    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-oauth-serving-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:33:04.452767 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.452771    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nt649\" (UniqueName: \"kubernetes.io/projected/6f665472-c2bd-48fb-b52b-a3597a9e521a-kube-api-access-nt649\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:33:04.452767 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.452779    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-oauth-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:33:04.453037 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.452788    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-service-ca\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:33:04.453037 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.452799    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:33:04.453037 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:04.452807    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f665472-c2bd-48fb-b52b-a3597a9e521a-console-serving-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:33:05.056207 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:05.056174    2581 generic.go:356] "Generic (PLEG): container finished" podID="6f665472-c2bd-48fb-b52b-a3597a9e521a" containerID="00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799" exitCode=0
Jul 01 19:33:05.056207 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:05.056213    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-587867dc88-vzb79" event={"ID":"6f665472-c2bd-48fb-b52b-a3597a9e521a","Type":"ContainerDied","Data":"00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799"}
Jul 01 19:33:05.056669 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:05.056217    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-587867dc88-vzb79"
Jul 01 19:33:05.056669 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:05.056249    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-587867dc88-vzb79" event={"ID":"6f665472-c2bd-48fb-b52b-a3597a9e521a","Type":"ContainerDied","Data":"94815079930c7720342335acfd3e8dd943e16b22f58078c41ee7275475ae0b8a"}
Jul 01 19:33:05.056669 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:05.056268    2581 scope.go:117] "RemoveContainer" containerID="00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799"
Jul 01 19:33:05.065267 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:05.065250    2581 scope.go:117] "RemoveContainer" containerID="00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799"
Jul 01 19:33:05.065528 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:33:05.065511    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799\": container with ID starting with 00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799 not found: ID does not exist" containerID="00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799"
Jul 01 19:33:05.065600 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:05.065537    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799"} err="failed to get container status \"00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799\": rpc error: code = NotFound desc = could not find container \"00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799\": container with ID starting with 00ab92482635fe43d3335c296ecd295b87d517f8e2d72b749499073952b37799 not found: ID does not exist"
Jul 01 19:33:05.077395 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:05.077370    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-587867dc88-vzb79"]
Jul 01 19:33:05.080185 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:05.080165    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-587867dc88-vzb79"]
Jul 01 19:33:05.216379 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:05.216346    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f665472-c2bd-48fb-b52b-a3597a9e521a" path="/var/lib/kubelet/pods/6f665472-c2bd-48fb-b52b-a3597a9e521a/volumes"
Jul 01 19:33:17.727974 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.727938    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"]
Jul 01 19:33:17.728381 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.728314    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f665472-c2bd-48fb-b52b-a3597a9e521a" containerName="console"
Jul 01 19:33:17.728381 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.728325    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f665472-c2bd-48fb-b52b-a3597a9e521a" containerName="console"
Jul 01 19:33:17.728381 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.728376    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f665472-c2bd-48fb-b52b-a3597a9e521a" containerName="console"
Jul 01 19:33:17.732742 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.732723    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"
Jul 01 19:33:17.737834 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.737812    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 01 19:33:17.737952 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.737835    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-z54t4\""
Jul 01 19:33:17.750563 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.750536    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"]
Jul 01 19:33:17.873180 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.873138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdgdg\" (UniqueName: \"kubernetes.io/projected/e0d32646-b847-4f72-95a8-0fc3944de114-kube-api-access-qdgdg\") pod \"seaweedfs-tls-custom-6644bf5d6c-dssrr\" (UID: \"e0d32646-b847-4f72-95a8-0fc3944de114\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"
Jul 01 19:33:17.873365 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.873223    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0d32646-b847-4f72-95a8-0fc3944de114-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-dssrr\" (UID: \"e0d32646-b847-4f72-95a8-0fc3944de114\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"
Jul 01 19:33:17.974632 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.974552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdgdg\" (UniqueName: \"kubernetes.io/projected/e0d32646-b847-4f72-95a8-0fc3944de114-kube-api-access-qdgdg\") pod \"seaweedfs-tls-custom-6644bf5d6c-dssrr\" (UID: \"e0d32646-b847-4f72-95a8-0fc3944de114\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"
Jul 01 19:33:17.974833 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.974674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0d32646-b847-4f72-95a8-0fc3944de114-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-dssrr\" (UID: \"e0d32646-b847-4f72-95a8-0fc3944de114\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"
Jul 01 19:33:17.975006 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.974991    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0d32646-b847-4f72-95a8-0fc3944de114-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-dssrr\" (UID: \"e0d32646-b847-4f72-95a8-0fc3944de114\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"
Jul 01 19:33:17.983470 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:17.983409    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdgdg\" (UniqueName: \"kubernetes.io/projected/e0d32646-b847-4f72-95a8-0fc3944de114-kube-api-access-qdgdg\") pod \"seaweedfs-tls-custom-6644bf5d6c-dssrr\" (UID: \"e0d32646-b847-4f72-95a8-0fc3944de114\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"
Jul 01 19:33:18.042267 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:18.042232    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"
Jul 01 19:33:18.177405 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:18.177381    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"]
Jul 01 19:33:18.179534 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:33:18.179506    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d32646_b847_4f72_95a8_0fc3944de114.slice/crio-4f2467d2292b7cd56582f3db0014af2fa0e5a2ca9c98169d63a98b056a3628dd WatchSource:0}: Error finding container 4f2467d2292b7cd56582f3db0014af2fa0e5a2ca9c98169d63a98b056a3628dd: Status 404 returned error can't find the container with id 4f2467d2292b7cd56582f3db0014af2fa0e5a2ca9c98169d63a98b056a3628dd
Jul 01 19:33:19.128409 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:19.128371    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr" event={"ID":"e0d32646-b847-4f72-95a8-0fc3944de114","Type":"ContainerStarted","Data":"4f2467d2292b7cd56582f3db0014af2fa0e5a2ca9c98169d63a98b056a3628dd"}
Jul 01 19:33:22.145015 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:22.144969    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr" event={"ID":"e0d32646-b847-4f72-95a8-0fc3944de114","Type":"ContainerStarted","Data":"bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006"}
Jul 01 19:33:22.163708 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:22.163652    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr" podStartSLOduration=1.343442519 podStartE2EDuration="5.163635569s" totalImagesPullingTime="3.82019305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:33:22.163080745 +0000 UTC m=+713.398206056" watchObservedRunningTime="2026-07-01 19:33:22.163635569 +0000 UTC m=+713.398760881"
Jul 01 19:33:23.361835 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:23.361803    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"]
Jul 01 19:33:24.153896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:24.153831    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr" podUID="e0d32646-b847-4f72-95a8-0fc3944de114" containerName="seaweedfs-tls-custom" containerID="cri-o://bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006" gracePeriod=30
Jul 01 19:33:52.705708 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:52.705682    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"
Jul 01 19:33:52.773118 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:52.773038    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0d32646-b847-4f72-95a8-0fc3944de114-data\") pod \"e0d32646-b847-4f72-95a8-0fc3944de114\" (UID: \"e0d32646-b847-4f72-95a8-0fc3944de114\") "
Jul 01 19:33:52.773118 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:52.773082    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdgdg\" (UniqueName: \"kubernetes.io/projected/e0d32646-b847-4f72-95a8-0fc3944de114-kube-api-access-qdgdg\") pod \"e0d32646-b847-4f72-95a8-0fc3944de114\" (UID: \"e0d32646-b847-4f72-95a8-0fc3944de114\") "
Jul 01 19:33:52.774313 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:52.774290    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d32646-b847-4f72-95a8-0fc3944de114-data" (OuterVolumeSpecName: "data") pod "e0d32646-b847-4f72-95a8-0fc3944de114" (UID: "e0d32646-b847-4f72-95a8-0fc3944de114"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:33:52.775138 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:52.775110    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0d32646-b847-4f72-95a8-0fc3944de114-kube-api-access-qdgdg" (OuterVolumeSpecName: "kube-api-access-qdgdg") pod "e0d32646-b847-4f72-95a8-0fc3944de114" (UID: "e0d32646-b847-4f72-95a8-0fc3944de114"). InnerVolumeSpecName "kube-api-access-qdgdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:33:52.874606 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:52.874544    2581 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0d32646-b847-4f72-95a8-0fc3944de114-data\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:33:52.874606 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:52.874600    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdgdg\" (UniqueName: \"kubernetes.io/projected/e0d32646-b847-4f72-95a8-0fc3944de114-kube-api-access-qdgdg\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:33:53.284851 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:53.284810    2581 generic.go:356] "Generic (PLEG): container finished" podID="e0d32646-b847-4f72-95a8-0fc3944de114" containerID="bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006" exitCode=0
Jul 01 19:33:53.285008 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:53.284870    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr" event={"ID":"e0d32646-b847-4f72-95a8-0fc3944de114","Type":"ContainerDied","Data":"bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006"}
Jul 01 19:33:53.285008 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:53.284927    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr" event={"ID":"e0d32646-b847-4f72-95a8-0fc3944de114","Type":"ContainerDied","Data":"4f2467d2292b7cd56582f3db0014af2fa0e5a2ca9c98169d63a98b056a3628dd"}
Jul 01 19:33:53.285008 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:53.284879    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"
Jul 01 19:33:53.285008 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:53.284948    2581 scope.go:117] "RemoveContainer" containerID="bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006"
Jul 01 19:33:53.294707 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:53.294689    2581 scope.go:117] "RemoveContainer" containerID="bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006"
Jul 01 19:33:53.294986 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:33:53.294965    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006\": container with ID starting with bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006 not found: ID does not exist" containerID="bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006"
Jul 01 19:33:53.295042 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:53.294995    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006"} err="failed to get container status \"bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006\": rpc error: code = NotFound desc = could not find container \"bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006\": container with ID starting with bf69db82183c5241e087bbee40719bd12d4d9632997d832c485c76a1fbc03006 not found: ID does not exist"
Jul 01 19:33:53.304648 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:53.304622    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"]
Jul 01 19:33:53.306523 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:53.306502    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-dssrr"]
Jul 01 19:33:55.215855 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:33:55.215823    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0d32646-b847-4f72-95a8-0fc3944de114" path="/var/lib/kubelet/pods/e0d32646-b847-4f72-95a8-0fc3944de114/volumes"
Jul 01 19:34:31.314871 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.314840    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"]
Jul 01 19:34:31.315270 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.315189    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d32646-b847-4f72-95a8-0fc3944de114" containerName="seaweedfs-tls-custom"
Jul 01 19:34:31.315270 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.315202    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d32646-b847-4f72-95a8-0fc3944de114" containerName="seaweedfs-tls-custom"
Jul 01 19:34:31.315270 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.315265    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d32646-b847-4f72-95a8-0fc3944de114" containerName="seaweedfs-tls-custom"
Jul 01 19:34:31.319887 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.319866    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.322862 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.322838    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-predictor-serving-cert\""
Jul 01 19:34:31.323006 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.322840    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 19:34:31.323176 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.323159    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 19:34:31.323692 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.323665    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 19:34:31.323855 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.323836    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 19:34:31.324298 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.324283    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2q28w\""
Jul 01 19:34:31.324298 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.324294    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-kube-rbac-proxy-sar-config\""
Jul 01 19:34:31.334379 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.334358    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"]
Jul 01 19:34:31.392687 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.392656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.392830 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.392698    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.392830 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.392790    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.392908 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.392830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.392908 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.392858    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jskh2\" (UniqueName: \"kubernetes.io/projected/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kube-api-access-jskh2\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.494271 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.494233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.494471 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.494284    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.494471 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.494333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.494471 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.494361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.494471 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.494396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jskh2\" (UniqueName: \"kubernetes.io/projected/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kube-api-access-jskh2\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.495098 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.495063    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.495216 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.495118    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.495216 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.495129    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.497214 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.497188    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.503554 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.503531    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jskh2\" (UniqueName: \"kubernetes.io/projected/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kube-api-access-jskh2\") pod \"isvc-sklearn-batcher-predictor-79ddf86687-gk9td\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.630252 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.630161    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:31.770506 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:31.770465    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"]
Jul 01 19:34:31.775277 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:34:31.775235    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d54c7ba_72c6_4ca4_88bc_693ca3e55274.slice/crio-140398fa9dc22d20984e6756d4426a4d9c1eef9c7b6e6eb47ac38f7b04ff2b50 WatchSource:0}: Error finding container 140398fa9dc22d20984e6756d4426a4d9c1eef9c7b6e6eb47ac38f7b04ff2b50: Status 404 returned error can't find the container with id 140398fa9dc22d20984e6756d4426a4d9c1eef9c7b6e6eb47ac38f7b04ff2b50
Jul 01 19:34:32.457513 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:32.457477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" event={"ID":"2d54c7ba-72c6-4ca4-88bc-693ca3e55274","Type":"ContainerStarted","Data":"140398fa9dc22d20984e6756d4426a4d9c1eef9c7b6e6eb47ac38f7b04ff2b50"}
Jul 01 19:34:35.473433 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:35.473394    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" event={"ID":"2d54c7ba-72c6-4ca4-88bc-693ca3e55274","Type":"ContainerStarted","Data":"537ab63b4e28d650fb1e2ead667dbb07bd0fffaa9e8de6edfaae6c8be915abf1"}
Jul 01 19:34:39.493970 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:39.493934    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerID="537ab63b4e28d650fb1e2ead667dbb07bd0fffaa9e8de6edfaae6c8be915abf1" exitCode=0
Jul 01 19:34:39.494336 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:39.494003    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" event={"ID":"2d54c7ba-72c6-4ca4-88bc-693ca3e55274","Type":"ContainerDied","Data":"537ab63b4e28d650fb1e2ead667dbb07bd0fffaa9e8de6edfaae6c8be915abf1"}
Jul 01 19:34:53.591476 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:53.591442    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" event={"ID":"2d54c7ba-72c6-4ca4-88bc-693ca3e55274","Type":"ContainerStarted","Data":"fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549"}
Jul 01 19:34:56.613492 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:56.613455    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" event={"ID":"2d54c7ba-72c6-4ca4-88bc-693ca3e55274","Type":"ContainerStarted","Data":"e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab"}
Jul 01 19:34:58.627379 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:58.627340    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" event={"ID":"2d54c7ba-72c6-4ca4-88bc-693ca3e55274","Type":"ContainerStarted","Data":"54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc"}
Jul 01 19:34:58.627785 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:58.627488    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:58.652745 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:58.652658    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podStartSLOduration=1.665513252 podStartE2EDuration="27.652621082s" totalImagesPullingTime="22.256440427s" totalInitContainerRuntime="3.730667403s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:34:31 +0000 UTC" imagePullSessionsCount=4 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:34:58.65007687 +0000 UTC m=+809.885202194" watchObservedRunningTime="2026-07-01 19:34:58.652621082 +0000 UTC m=+809.887746409"
Jul 01 19:34:59.632391 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:59.632349    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:59.632820 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:59.632426    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:34:59.633882 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:59.633845    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:34:59.634492 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:34:59.634466    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:35:00.637434 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:00.637396    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:35:00.637945 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:00.637863    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:35:00.641503 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:00.641483    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:35:01.642418 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:01.642379    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:35:01.643063 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:01.642799    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:35:11.643020 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:11.642974    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:35:11.643486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:11.643401    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:35:21.642927 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:21.642882    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:35:21.643486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:21.643321    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:35:31.643268 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:31.643224    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:35:31.643768 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:31.643744    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:35:41.642747 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:41.642703    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:35:41.643130 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:41.643098    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:35:51.642533 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:51.642484    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:35:51.642991 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:35:51.642965    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:36:01.642485 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:01.642429    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:36:01.643063 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:01.642944    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:36:03.210360 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:03.210315    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:36:03.210811 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:03.210683    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:36:13.215935 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:13.215909    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:36:13.216309 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:13.215999    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:36:26.412695 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.412660    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"]
Jul 01 19:36:26.413205 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.413147    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" containerID="cri-o://fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549" gracePeriod=30
Jul 01 19:36:26.413650 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.413621    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" containerID="cri-o://54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc" gracePeriod=30
Jul 01 19:36:26.414718 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.414075    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kube-rbac-proxy" containerID="cri-o://e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab" gracePeriod=30
Jul 01 19:36:26.449798 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.449767    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"]
Jul 01 19:36:26.453197 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.453176    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.457120 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.457095    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-predictor-serving-cert\""
Jul 01 19:36:26.457259 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.457130    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-kube-rbac-proxy-sar-config\""
Jul 01 19:36:26.473210 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.473182    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"]
Jul 01 19:36:26.495250 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.495213    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/494e11d4-5652-4373-8c71-525de25951e7-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.495525 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.495255    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/494e11d4-5652-4373-8c71-525de25951e7-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.495525 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.495330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/494e11d4-5652-4373-8c71-525de25951e7-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.495525 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.495426    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzw9q\" (UniqueName: \"kubernetes.io/projected/494e11d4-5652-4373-8c71-525de25951e7-kube-api-access-hzw9q\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.495525 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.495458    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/494e11d4-5652-4373-8c71-525de25951e7-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.596295 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.596257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/494e11d4-5652-4373-8c71-525de25951e7-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.596295 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.596300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/494e11d4-5652-4373-8c71-525de25951e7-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.596544 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.596318    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/494e11d4-5652-4373-8c71-525de25951e7-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.596544 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.596357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzw9q\" (UniqueName: \"kubernetes.io/projected/494e11d4-5652-4373-8c71-525de25951e7-kube-api-access-hzw9q\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.596544 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.596382    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/494e11d4-5652-4373-8c71-525de25951e7-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.596866 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.596845    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/494e11d4-5652-4373-8c71-525de25951e7-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.597069 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.597051    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/494e11d4-5652-4373-8c71-525de25951e7-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.597113 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.597091    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/494e11d4-5652-4373-8c71-525de25951e7-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.598881 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.598857    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/494e11d4-5652-4373-8c71-525de25951e7-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.610093 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.610062    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzw9q\" (UniqueName: \"kubernetes.io/projected/494e11d4-5652-4373-8c71-525de25951e7-kube-api-access-hzw9q\") pod \"isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.763634 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.763563    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:26.901532 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:26.901507    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"]
Jul 01 19:36:26.903807 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:36:26.903781    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494e11d4_5652_4373_8c71_525de25951e7.slice/crio-99c5b610a1c4472147409797012a2f0e1f4d7ac58c686d23a47b69975b07f7bf WatchSource:0}: Error finding container 99c5b610a1c4472147409797012a2f0e1f4d7ac58c686d23a47b69975b07f7bf: Status 404 returned error can't find the container with id 99c5b610a1c4472147409797012a2f0e1f4d7ac58c686d23a47b69975b07f7bf
Jul 01 19:36:27.050528 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:27.050436    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerID="e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab" exitCode=2
Jul 01 19:36:27.050528 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:27.050475    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" event={"ID":"2d54c7ba-72c6-4ca4-88bc-693ca3e55274","Type":"ContainerDied","Data":"e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab"}
Jul 01 19:36:27.052102 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:27.052078    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" event={"ID":"494e11d4-5652-4373-8c71-525de25951e7","Type":"ContainerStarted","Data":"8d4c0e3efe1be77fc44d38480e8fc92526e506d25ec2588f3430dab09074baa5"}
Jul 01 19:36:27.052228 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:27.052139    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" event={"ID":"494e11d4-5652-4373-8c71-525de25951e7","Type":"ContainerStarted","Data":"99c5b610a1c4472147409797012a2f0e1f4d7ac58c686d23a47b69975b07f7bf"}
Jul 01 19:36:29.203551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:29.203523    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:36:29.204401 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:29.204379    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:36:30.638691 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:30.638652    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused"
Jul 01 19:36:31.072511 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:31.072482    2581 generic.go:356] "Generic (PLEG): container finished" podID="494e11d4-5652-4373-8c71-525de25951e7" containerID="8d4c0e3efe1be77fc44d38480e8fc92526e506d25ec2588f3430dab09074baa5" exitCode=0
Jul 01 19:36:31.072670 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:31.072558    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" event={"ID":"494e11d4-5652-4373-8c71-525de25951e7","Type":"ContainerDied","Data":"8d4c0e3efe1be77fc44d38480e8fc92526e506d25ec2588f3430dab09074baa5"}
Jul 01 19:36:31.075309 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:31.075284    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerID="fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549" exitCode=0
Jul 01 19:36:31.075384 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:31.075360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" event={"ID":"2d54c7ba-72c6-4ca4-88bc-693ca3e55274","Type":"ContainerDied","Data":"fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549"}
Jul 01 19:36:32.083810 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:32.083780    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" event={"ID":"494e11d4-5652-4373-8c71-525de25951e7","Type":"ContainerStarted","Data":"5f0dd03a98202a5ee8fae9e85d524bb3bda74dba0a80ae04e553dea198957eb5"}
Jul 01 19:36:32.084231 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:32.083855    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" event={"ID":"494e11d4-5652-4373-8c71-525de25951e7","Type":"ContainerStarted","Data":"9b14e63314eb0fe6be53a55dac0d2b35f78c8f25a7f46cf13dff360189128940"}
Jul 01 19:36:32.084231 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:32.083887    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" event={"ID":"494e11d4-5652-4373-8c71-525de25951e7","Type":"ContainerStarted","Data":"38e8c0e3e113b3283265eae1f6b0e1b3ee773bb05df92a9e101f103ad148c888"}
Jul 01 19:36:32.084231 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:32.084206    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:32.084375 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:32.084279    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:32.085678 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:32.085650    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:36:32.112290 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:32.112247    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podStartSLOduration=2.46791146 podStartE2EDuration="6.112234096s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.644322636s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:36:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:36:32.109267722 +0000 UTC m=+903.344393059" watchObservedRunningTime="2026-07-01 19:36:32.112234096 +0000 UTC m=+903.347359408"
Jul 01 19:36:33.090860 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:33.090823    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:33.091447 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:33.090940    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:36:33.091879 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:33.091845    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:36:33.211006 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:33.210965    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:36:33.211323 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:33.211297    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:36:34.096054 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:34.096006    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:36:34.096473 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:34.096449    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:36:35.638448 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:35.638398    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused"
Jul 01 19:36:39.101455 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:39.101420    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:36:39.102176 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:39.102145    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:36:39.102424 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:39.102391    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:36:40.638647 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:40.638603    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused"
Jul 01 19:36:40.639101 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:40.638722    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:36:43.210557 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:43.210507    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:36:43.212224 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:43.210805    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:36:45.637731 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:45.637689    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused"
Jul 01 19:36:49.102894 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:49.102846    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:36:49.103297 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:49.103255    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:36:50.638066 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:50.638023    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused"
Jul 01 19:36:53.211090 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:53.211038    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 01 19:36:53.211523 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:53.211425    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:36:53.215540 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:53.215518    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:36:53.215691 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:53.215638    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:36:55.637753 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:55.637710    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused"
Jul 01 19:36:56.573334 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.573304    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:36:56.674403 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.674302    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-cabundle-cert\") pod \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") "
Jul 01 19:36:56.674403 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.674351    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kserve-provision-location\") pod \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") "
Jul 01 19:36:56.674403 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.674403    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kube-rbac-proxy-sar-config\") pod \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") "
Jul 01 19:36:56.675009 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.674424    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-proxy-tls\") pod \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") "
Jul 01 19:36:56.675009 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.674461    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jskh2\" (UniqueName: \"kubernetes.io/projected/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kube-api-access-jskh2\") pod \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\" (UID: \"2d54c7ba-72c6-4ca4-88bc-693ca3e55274\") "
Jul 01 19:36:56.675009 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.674771    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "2d54c7ba-72c6-4ca4-88bc-693ca3e55274" (UID: "2d54c7ba-72c6-4ca4-88bc-693ca3e55274"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:36:56.675009 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.674810    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2d54c7ba-72c6-4ca4-88bc-693ca3e55274" (UID: "2d54c7ba-72c6-4ca4-88bc-693ca3e55274"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:36:56.675009 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.674828    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2d54c7ba-72c6-4ca4-88bc-693ca3e55274" (UID: "2d54c7ba-72c6-4ca4-88bc-693ca3e55274"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:36:56.676864 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.676832    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kube-api-access-jskh2" (OuterVolumeSpecName: "kube-api-access-jskh2") pod "2d54c7ba-72c6-4ca4-88bc-693ca3e55274" (UID: "2d54c7ba-72c6-4ca4-88bc-693ca3e55274"). InnerVolumeSpecName "kube-api-access-jskh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:36:56.677085 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.676867    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2d54c7ba-72c6-4ca4-88bc-693ca3e55274" (UID: "2d54c7ba-72c6-4ca4-88bc-693ca3e55274"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:36:56.776191 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.776147    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-cabundle-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:36:56.776191 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.776176    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kserve-provision-location\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:36:56.776191 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.776187    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:36:56.776191 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.776196    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-proxy-tls\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:36:56.776191 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:56.776204    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jskh2\" (UniqueName: \"kubernetes.io/projected/2d54c7ba-72c6-4ca4-88bc-693ca3e55274-kube-api-access-jskh2\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:36:57.217523 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.217488    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerID="54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc" exitCode=0
Jul 01 19:36:57.217523 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.217519    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" event={"ID":"2d54c7ba-72c6-4ca4-88bc-693ca3e55274","Type":"ContainerDied","Data":"54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc"}
Jul 01 19:36:57.217762 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.217554    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"
Jul 01 19:36:57.217762 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.217605    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td" event={"ID":"2d54c7ba-72c6-4ca4-88bc-693ca3e55274","Type":"ContainerDied","Data":"140398fa9dc22d20984e6756d4426a4d9c1eef9c7b6e6eb47ac38f7b04ff2b50"}
Jul 01 19:36:57.217762 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.217628    2581 scope.go:117] "RemoveContainer" containerID="54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc"
Jul 01 19:36:57.226801 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.226785    2581 scope.go:117] "RemoveContainer" containerID="e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab"
Jul 01 19:36:57.234118 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.234100    2581 scope.go:117] "RemoveContainer" containerID="fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549"
Jul 01 19:36:57.242117 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.242090    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"]
Jul 01 19:36:57.242697 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.242678    2581 scope.go:117] "RemoveContainer" containerID="537ab63b4e28d650fb1e2ead667dbb07bd0fffaa9e8de6edfaae6c8be915abf1"
Jul 01 19:36:57.245868 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.245849    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-79ddf86687-gk9td"]
Jul 01 19:36:57.250355 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.250318    2581 scope.go:117] "RemoveContainer" containerID="54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc"
Jul 01 19:36:57.250621 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:36:57.250598    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc\": container with ID starting with 54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc not found: ID does not exist" containerID="54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc"
Jul 01 19:36:57.250717 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.250627    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc"} err="failed to get container status \"54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc\": rpc error: code = NotFound desc = could not find container \"54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc\": container with ID starting with 54227ea05b436db3192684b8ea25b7105092781923c4610c2e4f409249093bcc not found: ID does not exist"
Jul 01 19:36:57.250717 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.250647    2581 scope.go:117] "RemoveContainer" containerID="e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab"
Jul 01 19:36:57.250893 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:36:57.250877    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab\": container with ID starting with e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab not found: ID does not exist" containerID="e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab"
Jul 01 19:36:57.250934 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.250899    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab"} err="failed to get container status \"e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab\": rpc error: code = NotFound desc = could not find container \"e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab\": container with ID starting with e0596902f03ab97ee570accf4d9f6859861fb1c661ab43dc0c41623d5df319ab not found: ID does not exist"
Jul 01 19:36:57.250934 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.250915    2581 scope.go:117] "RemoveContainer" containerID="fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549"
Jul 01 19:36:57.251114 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:36:57.251097    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549\": container with ID starting with fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549 not found: ID does not exist" containerID="fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549"
Jul 01 19:36:57.251152 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.251119    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549"} err="failed to get container status \"fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549\": rpc error: code = NotFound desc = could not find container \"fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549\": container with ID starting with fef1b4744b1dcd07394796a7cfab9c96289c67f80f9cfa88d5593ef3be956549 not found: ID does not exist"
Jul 01 19:36:57.251152 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.251132    2581 scope.go:117] "RemoveContainer" containerID="537ab63b4e28d650fb1e2ead667dbb07bd0fffaa9e8de6edfaae6c8be915abf1"
Jul 01 19:36:57.251317 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:36:57.251300    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"537ab63b4e28d650fb1e2ead667dbb07bd0fffaa9e8de6edfaae6c8be915abf1\": container with ID starting with 537ab63b4e28d650fb1e2ead667dbb07bd0fffaa9e8de6edfaae6c8be915abf1 not found: ID does not exist" containerID="537ab63b4e28d650fb1e2ead667dbb07bd0fffaa9e8de6edfaae6c8be915abf1"
Jul 01 19:36:57.251356 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:57.251319    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"537ab63b4e28d650fb1e2ead667dbb07bd0fffaa9e8de6edfaae6c8be915abf1"} err="failed to get container status \"537ab63b4e28d650fb1e2ead667dbb07bd0fffaa9e8de6edfaae6c8be915abf1\": rpc error: code = NotFound desc = could not find container \"537ab63b4e28d650fb1e2ead667dbb07bd0fffaa9e8de6edfaae6c8be915abf1\": container with ID starting with 537ab63b4e28d650fb1e2ead667dbb07bd0fffaa9e8de6edfaae6c8be915abf1 not found: ID does not exist"
Jul 01 19:36:59.102495 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:59.102460    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:36:59.102977 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:59.102953    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:36:59.215889 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:36:59.215856    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" path="/var/lib/kubelet/pods/2d54c7ba-72c6-4ca4-88bc-693ca3e55274/volumes"
Jul 01 19:37:09.102382 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:37:09.102334    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:37:09.102948 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:37:09.102807    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:37:19.102309 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:37:19.102262    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:37:19.102842 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:37:19.102750    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:37:29.102660 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:37:29.102608    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:37:29.103108 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:37:29.103082    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:37:39.102633 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:37:39.102570    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:37:39.103079 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:37:39.103051    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:37:49.102702 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:37:49.102671    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:37:49.103086 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:37:49.102772    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:38:01.633898 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:01.633865    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"]
Jul 01 19:38:01.634451 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:01.634136    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" containerID="cri-o://5f0dd03a98202a5ee8fae9e85d524bb3bda74dba0a80ae04e553dea198957eb5" gracePeriod=30
Jul 01 19:38:01.634451 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:01.634136    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" containerID="cri-o://38e8c0e3e113b3283265eae1f6b0e1b3ee773bb05df92a9e101f103ad148c888" gracePeriod=30
Jul 01 19:38:01.634451 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:01.634185    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kube-rbac-proxy" containerID="cri-o://9b14e63314eb0fe6be53a55dac0d2b35f78c8f25a7f46cf13dff360189128940" gracePeriod=30
Jul 01 19:38:02.526294 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:02.526257    2581 generic.go:356] "Generic (PLEG): container finished" podID="494e11d4-5652-4373-8c71-525de25951e7" containerID="9b14e63314eb0fe6be53a55dac0d2b35f78c8f25a7f46cf13dff360189128940" exitCode=2
Jul 01 19:38:02.526487 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:02.526312    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" event={"ID":"494e11d4-5652-4373-8c71-525de25951e7","Type":"ContainerDied","Data":"9b14e63314eb0fe6be53a55dac0d2b35f78c8f25a7f46cf13dff360189128940"}
Jul 01 19:38:04.096967 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:04.096921    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.36:8643/healthz\": dial tcp 10.132.0.36:8643: connect: connection refused"
Jul 01 19:38:06.550875 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:06.550791    2581 generic.go:356] "Generic (PLEG): container finished" podID="494e11d4-5652-4373-8c71-525de25951e7" containerID="38e8c0e3e113b3283265eae1f6b0e1b3ee773bb05df92a9e101f103ad148c888" exitCode=0
Jul 01 19:38:06.550875 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:06.550866    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" event={"ID":"494e11d4-5652-4373-8c71-525de25951e7","Type":"ContainerDied","Data":"38e8c0e3e113b3283265eae1f6b0e1b3ee773bb05df92a9e101f103ad148c888"}
Jul 01 19:38:09.096676 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:09.096638    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.36:8643/healthz\": dial tcp 10.132.0.36:8643: connect: connection refused"
Jul 01 19:38:09.101968 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:09.101940    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:38:09.102300 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:09.102281    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:38:14.096870 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:14.096823    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.36:8643/healthz\": dial tcp 10.132.0.36:8643: connect: connection refused"
Jul 01 19:38:14.097342 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:14.096968    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:38:19.096750 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:19.096701    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.36:8643/healthz\": dial tcp 10.132.0.36:8643: connect: connection refused"
Jul 01 19:38:19.102510 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:19.102473    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:38:19.102847 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:19.102816    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:38:21.670363 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.670327    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"]
Jul 01 19:38:21.670783 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.670724    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container"
Jul 01 19:38:21.670783 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.670735    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container"
Jul 01 19:38:21.670783 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.670749    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="storage-initializer"
Jul 01 19:38:21.670783 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.670755    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="storage-initializer"
Jul 01 19:38:21.670783 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.670770    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kube-rbac-proxy"
Jul 01 19:38:21.670783 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.670776    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kube-rbac-proxy"
Jul 01 19:38:21.670783 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.670786    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent"
Jul 01 19:38:21.671001 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.670792    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent"
Jul 01 19:38:21.671001 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.670861    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kube-rbac-proxy"
Jul 01 19:38:21.671001 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.670868    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="kserve-container"
Jul 01 19:38:21.671001 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.670876    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d54c7ba-72c6-4ca4-88bc-693ca3e55274" containerName="agent"
Jul 01 19:38:21.674372 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.674351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.677849 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.677823    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-predictor-serving-cert\""
Jul 01 19:38:21.679110 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.679062    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-kube-rbac-proxy-sar-config\""
Jul 01 19:38:21.693787 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.693762    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"]
Jul 01 19:38:21.829807 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.829768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/944a456c-d53c-434a-850c-b7418f7f6116-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.829807 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.829812    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/944a456c-d53c-434a-850c-b7418f7f6116-cabundle-cert\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.830009 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.829834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/944a456c-d53c-434a-850c-b7418f7f6116-proxy-tls\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.830009 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.829857    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv5v5\" (UniqueName: \"kubernetes.io/projected/944a456c-d53c-434a-850c-b7418f7f6116-kube-api-access-lv5v5\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.830009 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.829875    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/944a456c-d53c-434a-850c-b7418f7f6116-kserve-provision-location\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.930867 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.930769    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/944a456c-d53c-434a-850c-b7418f7f6116-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.930867 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.930818    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/944a456c-d53c-434a-850c-b7418f7f6116-cabundle-cert\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.930867 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.930866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/944a456c-d53c-434a-850c-b7418f7f6116-proxy-tls\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.931153 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.930972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv5v5\" (UniqueName: \"kubernetes.io/projected/944a456c-d53c-434a-850c-b7418f7f6116-kube-api-access-lv5v5\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.931153 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.931015    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/944a456c-d53c-434a-850c-b7418f7f6116-kserve-provision-location\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.931451 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.931423    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/944a456c-d53c-434a-850c-b7418f7f6116-kserve-provision-location\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.931614 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.931513    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/944a456c-d53c-434a-850c-b7418f7f6116-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.931704 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.931666    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/944a456c-d53c-434a-850c-b7418f7f6116-cabundle-cert\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.933395 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.933376    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/944a456c-d53c-434a-850c-b7418f7f6116-proxy-tls\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.940234 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.940214    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv5v5\" (UniqueName: \"kubernetes.io/projected/944a456c-d53c-434a-850c-b7418f7f6116-kube-api-access-lv5v5\") pod \"isvc-logger-predictor-5597c66d9b-tqs97\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:21.985321 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:21.985273    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:22.124033 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:22.121681    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"]
Jul 01 19:38:22.127468 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:38:22.127421    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod944a456c_d53c_434a_850c_b7418f7f6116.slice/crio-b676aaab3733e3be782158b9c9a1656afbb1fca03db69e36e4a567e07380ab67 WatchSource:0}: Error finding container b676aaab3733e3be782158b9c9a1656afbb1fca03db69e36e4a567e07380ab67: Status 404 returned error can't find the container with id b676aaab3733e3be782158b9c9a1656afbb1fca03db69e36e4a567e07380ab67
Jul 01 19:38:22.627333 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:22.627303    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" event={"ID":"944a456c-d53c-434a-850c-b7418f7f6116","Type":"ContainerStarted","Data":"9dc17f5b35156bd51de91f8508a35faad8c2b7d8e5f9fb1ba93a133ac24d8668"}
Jul 01 19:38:22.627514 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:22.627365    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" event={"ID":"944a456c-d53c-434a-850c-b7418f7f6116","Type":"ContainerStarted","Data":"b676aaab3733e3be782158b9c9a1656afbb1fca03db69e36e4a567e07380ab67"}
Jul 01 19:38:24.096929 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:24.096889    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.36:8643/healthz\": dial tcp 10.132.0.36:8643: connect: connection refused"
Jul 01 19:38:26.648033 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:26.648001    2581 generic.go:356] "Generic (PLEG): container finished" podID="944a456c-d53c-434a-850c-b7418f7f6116" containerID="9dc17f5b35156bd51de91f8508a35faad8c2b7d8e5f9fb1ba93a133ac24d8668" exitCode=0
Jul 01 19:38:26.648387 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:26.648069    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" event={"ID":"944a456c-d53c-434a-850c-b7418f7f6116","Type":"ContainerDied","Data":"9dc17f5b35156bd51de91f8508a35faad8c2b7d8e5f9fb1ba93a133ac24d8668"}
Jul 01 19:38:27.656521 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:27.656483    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" event={"ID":"944a456c-d53c-434a-850c-b7418f7f6116","Type":"ContainerStarted","Data":"eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9"}
Jul 01 19:38:27.656954 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:27.656566    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" event={"ID":"944a456c-d53c-434a-850c-b7418f7f6116","Type":"ContainerStarted","Data":"f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796"}
Jul 01 19:38:27.656954 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:27.656619    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" event={"ID":"944a456c-d53c-434a-850c-b7418f7f6116","Type":"ContainerStarted","Data":"8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3"}
Jul 01 19:38:27.656954 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:27.656897    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:27.657097 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:27.657078    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:27.658528 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:27.658501    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:38:27.679072 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:27.679024    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podStartSLOduration=3.06911728 podStartE2EDuration="6.679009666s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.609892386s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:38:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:38:27.677064238 +0000 UTC m=+1018.912189594" watchObservedRunningTime="2026-07-01 19:38:27.679009666 +0000 UTC m=+1018.914134977"
Jul 01 19:38:28.662100 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:28.662060    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:28.662568 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:28.662060    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:38:28.663221 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:28.663188    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:38:29.096628 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:29.096563    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.36:8643/healthz\": dial tcp 10.132.0.36:8643: connect: connection refused"
Jul 01 19:38:29.102126 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:29.102093    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:5000: connect: connection refused"
Jul 01 19:38:29.102240 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:29.102231    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:38:29.102494 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:29.102464    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:38:29.102624 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:29.102609    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:38:29.667465 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:29.667417    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:38:29.667904 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:29.667795    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:38:31.680959 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.680923    2581 generic.go:356] "Generic (PLEG): container finished" podID="494e11d4-5652-4373-8c71-525de25951e7" containerID="5f0dd03a98202a5ee8fae9e85d524bb3bda74dba0a80ae04e553dea198957eb5" exitCode=0
Jul 01 19:38:31.681433 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.680997    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" event={"ID":"494e11d4-5652-4373-8c71-525de25951e7","Type":"ContainerDied","Data":"5f0dd03a98202a5ee8fae9e85d524bb3bda74dba0a80ae04e553dea198957eb5"}
Jul 01 19:38:31.800565 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.800537    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:38:31.929118 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.929026    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/494e11d4-5652-4373-8c71-525de25951e7-proxy-tls\") pod \"494e11d4-5652-4373-8c71-525de25951e7\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") "
Jul 01 19:38:31.929118 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.929103    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/494e11d4-5652-4373-8c71-525de25951e7-kube-rbac-proxy-sar-config\") pod \"494e11d4-5652-4373-8c71-525de25951e7\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") "
Jul 01 19:38:31.929118 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.929121    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/494e11d4-5652-4373-8c71-525de25951e7-kserve-provision-location\") pod \"494e11d4-5652-4373-8c71-525de25951e7\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") "
Jul 01 19:38:31.929338 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.929166    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzw9q\" (UniqueName: \"kubernetes.io/projected/494e11d4-5652-4373-8c71-525de25951e7-kube-api-access-hzw9q\") pod \"494e11d4-5652-4373-8c71-525de25951e7\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") "
Jul 01 19:38:31.929338 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.929258    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/494e11d4-5652-4373-8c71-525de25951e7-cabundle-cert\") pod \"494e11d4-5652-4373-8c71-525de25951e7\" (UID: \"494e11d4-5652-4373-8c71-525de25951e7\") "
Jul 01 19:38:31.929493 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.929463    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/494e11d4-5652-4373-8c71-525de25951e7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "494e11d4-5652-4373-8c71-525de25951e7" (UID: "494e11d4-5652-4373-8c71-525de25951e7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:38:31.929559 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.929479    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/494e11d4-5652-4373-8c71-525de25951e7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "494e11d4-5652-4373-8c71-525de25951e7" (UID: "494e11d4-5652-4373-8c71-525de25951e7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:38:31.929786 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.929756    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/494e11d4-5652-4373-8c71-525de25951e7-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "494e11d4-5652-4373-8c71-525de25951e7" (UID: "494e11d4-5652-4373-8c71-525de25951e7"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:38:31.931311 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.931286    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/494e11d4-5652-4373-8c71-525de25951e7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "494e11d4-5652-4373-8c71-525de25951e7" (UID: "494e11d4-5652-4373-8c71-525de25951e7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:38:31.931311 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:31.931298    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/494e11d4-5652-4373-8c71-525de25951e7-kube-api-access-hzw9q" (OuterVolumeSpecName: "kube-api-access-hzw9q") pod "494e11d4-5652-4373-8c71-525de25951e7" (UID: "494e11d4-5652-4373-8c71-525de25951e7"). InnerVolumeSpecName "kube-api-access-hzw9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:38:32.031044 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.031003    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/494e11d4-5652-4373-8c71-525de25951e7-cabundle-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:38:32.031044 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.031041    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/494e11d4-5652-4373-8c71-525de25951e7-proxy-tls\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:38:32.031044 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.031051    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/494e11d4-5652-4373-8c71-525de25951e7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:38:32.031326 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.031061    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/494e11d4-5652-4373-8c71-525de25951e7-kserve-provision-location\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:38:32.031326 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.031070    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzw9q\" (UniqueName: \"kubernetes.io/projected/494e11d4-5652-4373-8c71-525de25951e7-kube-api-access-hzw9q\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:38:32.687594 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.687542    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs" event={"ID":"494e11d4-5652-4373-8c71-525de25951e7","Type":"ContainerDied","Data":"99c5b610a1c4472147409797012a2f0e1f4d7ac58c686d23a47b69975b07f7bf"}
Jul 01 19:38:32.688033 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.687609    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"
Jul 01 19:38:32.688033 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.687652    2581 scope.go:117] "RemoveContainer" containerID="5f0dd03a98202a5ee8fae9e85d524bb3bda74dba0a80ae04e553dea198957eb5"
Jul 01 19:38:32.696819 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.696801    2581 scope.go:117] "RemoveContainer" containerID="9b14e63314eb0fe6be53a55dac0d2b35f78c8f25a7f46cf13dff360189128940"
Jul 01 19:38:32.704377 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.704357    2581 scope.go:117] "RemoveContainer" containerID="38e8c0e3e113b3283265eae1f6b0e1b3ee773bb05df92a9e101f103ad148c888"
Jul 01 19:38:32.710606 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.710563    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"]
Jul 01 19:38:32.712884 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.712866    2581 scope.go:117] "RemoveContainer" containerID="8d4c0e3efe1be77fc44d38480e8fc92526e506d25ec2588f3430dab09074baa5"
Jul 01 19:38:32.714560 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:32.714541    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-55858799cd-tsrbs"]
Jul 01 19:38:33.215638 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:33.215607    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="494e11d4-5652-4373-8c71-525de25951e7" path="/var/lib/kubelet/pods/494e11d4-5652-4373-8c71-525de25951e7/volumes"
Jul 01 19:38:34.673084 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:34.673057    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:38:34.673678 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:34.673650    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:38:34.674034 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:34.674010    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:38:44.673740 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:44.673686    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:38:44.674244 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:44.674158    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:38:54.673824 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:54.673767    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:38:54.674326 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:38:54.674134    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:39:04.673549 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:04.673499    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:39:04.674072 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:04.673958    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:39:14.673846 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:14.673796    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:39:14.674291 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:14.674193    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:39:24.674400 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:24.674359    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:39:24.674945 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:24.674800    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:39:34.673854 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:34.673812    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:39:34.674319 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:34.674295    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:39:44.210727 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:44.210695    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:39:44.211155 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:44.210778    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:39:56.974625 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.974543    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"]
Jul 01 19:39:56.975229 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.975158    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="storage-initializer"
Jul 01 19:39:56.975229 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.975179    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="storage-initializer"
Jul 01 19:39:56.975229 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.975195    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent"
Jul 01 19:39:56.975229 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.975204    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent"
Jul 01 19:39:56.975229 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.975228    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kube-rbac-proxy"
Jul 01 19:39:56.975493 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.975238    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kube-rbac-proxy"
Jul 01 19:39:56.975493 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.975265    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container"
Jul 01 19:39:56.975493 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.975306    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container"
Jul 01 19:39:56.975493 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.975434    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kube-rbac-proxy"
Jul 01 19:39:56.975493 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.975450    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="agent"
Jul 01 19:39:56.975493 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.975462    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="494e11d4-5652-4373-8c71-525de25951e7" containerName="kserve-container"
Jul 01 19:39:56.978803 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.978781    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"]
Jul 01 19:39:56.978939 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.978920    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:56.979383 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.979141    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" containerID="cri-o://8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3" gracePeriod=30
Jul 01 19:39:56.979383 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.979200    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" containerID="cri-o://eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9" gracePeriod=30
Jul 01 19:39:56.979383 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.979199    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kube-rbac-proxy" containerID="cri-o://f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796" gracePeriod=30
Jul 01 19:39:56.982329 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.982290    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-kube-rbac-proxy-sar-config\""
Jul 01 19:39:56.982878 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.982850    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-predictor-serving-cert\""
Jul 01 19:39:56.992310 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:56.992279    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"]
Jul 01 19:39:57.083286 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.083254    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d989ba85-7454-4631-98f2-3d8dcd50aeeb-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.083467 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.083293    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lcf5\" (UniqueName: \"kubernetes.io/projected/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kube-api-access-7lcf5\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.083467 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.083322    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.083467 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.083364    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d989ba85-7454-4631-98f2-3d8dcd50aeeb-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.083467 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.083437    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.096796 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.096766    2581 generic.go:356] "Generic (PLEG): container finished" podID="944a456c-d53c-434a-850c-b7418f7f6116" containerID="f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796" exitCode=2
Jul 01 19:39:57.096931 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.096833    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" event={"ID":"944a456c-d53c-434a-850c-b7418f7f6116","Type":"ContainerDied","Data":"f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796"}
Jul 01 19:39:57.184728 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.184695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.184927 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.184751    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d989ba85-7454-4631-98f2-3d8dcd50aeeb-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.184927 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.184772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lcf5\" (UniqueName: \"kubernetes.io/projected/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kube-api-access-7lcf5\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.184927 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.184799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.184927 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.184815    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d989ba85-7454-4631-98f2-3d8dcd50aeeb-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.185310 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.185281    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.185485 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.185463    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.185547 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.185500    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d989ba85-7454-4631-98f2-3d8dcd50aeeb-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.187363 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.187342    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d989ba85-7454-4631-98f2-3d8dcd50aeeb-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.193676 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.193644    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lcf5\" (UniqueName: \"kubernetes.io/projected/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kube-api-access-7lcf5\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.303395 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.303310    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:39:57.440342 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:57.440315    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"]
Jul 01 19:39:57.441901 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:39:57.441877    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd989ba85_7454_4631_98f2_3d8dcd50aeeb.slice/crio-3c1549e0802351bb2907c39ec961c11cab52f7b0c2a576a0580d1f31616b5c04 WatchSource:0}: Error finding container 3c1549e0802351bb2907c39ec961c11cab52f7b0c2a576a0580d1f31616b5c04: Status 404 returned error can't find the container with id 3c1549e0802351bb2907c39ec961c11cab52f7b0c2a576a0580d1f31616b5c04
Jul 01 19:39:58.109096 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:58.109055    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" event={"ID":"d989ba85-7454-4631-98f2-3d8dcd50aeeb","Type":"ContainerStarted","Data":"7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4"}
Jul 01 19:39:58.109484 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:58.109162    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" event={"ID":"d989ba85-7454-4631-98f2-3d8dcd50aeeb","Type":"ContainerStarted","Data":"3c1549e0802351bb2907c39ec961c11cab52f7b0c2a576a0580d1f31616b5c04"}
Jul 01 19:39:59.667790 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:39:59.667744    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 01 19:40:02.131990 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:02.131899    2581 generic.go:356] "Generic (PLEG): container finished" podID="944a456c-d53c-434a-850c-b7418f7f6116" containerID="8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3" exitCode=0
Jul 01 19:40:02.131990 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:02.131971    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" event={"ID":"944a456c-d53c-434a-850c-b7418f7f6116","Type":"ContainerDied","Data":"8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3"}
Jul 01 19:40:04.210288 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:04.210234    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:40:04.210859 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:04.210612    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:40:04.668420 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:04.668313    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 01 19:40:09.668048 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:09.668001    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 01 19:40:09.668616 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:09.668168    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:40:14.210184 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:14.210127    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:40:14.210668 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:14.210521    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:40:14.668514 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:14.668415    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 01 19:40:19.668121 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:19.668066    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 01 19:40:24.210459 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:24.210395    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 01 19:40:24.211001 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:24.210549    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:40:24.211001 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:24.210699    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 01 19:40:24.211001 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:24.210823    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:40:24.668604 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:24.668474    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 01 19:40:27.148931 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.148901    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:40:27.254662 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.254548    2581 generic.go:356] "Generic (PLEG): container finished" podID="944a456c-d53c-434a-850c-b7418f7f6116" containerID="eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9" exitCode=0
Jul 01 19:40:27.254662 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.254634    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"
Jul 01 19:40:27.254891 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.254632    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" event={"ID":"944a456c-d53c-434a-850c-b7418f7f6116","Type":"ContainerDied","Data":"eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9"}
Jul 01 19:40:27.254891 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.254783    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97" event={"ID":"944a456c-d53c-434a-850c-b7418f7f6116","Type":"ContainerDied","Data":"b676aaab3733e3be782158b9c9a1656afbb1fca03db69e36e4a567e07380ab67"}
Jul 01 19:40:27.254891 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.254823    2581 scope.go:117] "RemoveContainer" containerID="eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9"
Jul 01 19:40:27.256156 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.256137    2581 generic.go:356] "Generic (PLEG): container finished" podID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerID="7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4" exitCode=0
Jul 01 19:40:27.256252 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.256209    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" event={"ID":"d989ba85-7454-4631-98f2-3d8dcd50aeeb","Type":"ContainerDied","Data":"7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4"}
Jul 01 19:40:27.257563 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.257548    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 19:40:27.258977 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.258952    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lv5v5\" (UniqueName: \"kubernetes.io/projected/944a456c-d53c-434a-850c-b7418f7f6116-kube-api-access-lv5v5\") pod \"944a456c-d53c-434a-850c-b7418f7f6116\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") "
Jul 01 19:40:27.259073 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.259015    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/944a456c-d53c-434a-850c-b7418f7f6116-kserve-provision-location\") pod \"944a456c-d53c-434a-850c-b7418f7f6116\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") "
Jul 01 19:40:27.259073 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.259059    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/944a456c-d53c-434a-850c-b7418f7f6116-cabundle-cert\") pod \"944a456c-d53c-434a-850c-b7418f7f6116\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") "
Jul 01 19:40:27.259165 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.259112    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/944a456c-d53c-434a-850c-b7418f7f6116-kube-rbac-proxy-sar-config\") pod \"944a456c-d53c-434a-850c-b7418f7f6116\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") "
Jul 01 19:40:27.259209 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.259180    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/944a456c-d53c-434a-850c-b7418f7f6116-proxy-tls\") pod \"944a456c-d53c-434a-850c-b7418f7f6116\" (UID: \"944a456c-d53c-434a-850c-b7418f7f6116\") "
Jul 01 19:40:27.259383 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.259357    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/944a456c-d53c-434a-850c-b7418f7f6116-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "944a456c-d53c-434a-850c-b7418f7f6116" (UID: "944a456c-d53c-434a-850c-b7418f7f6116"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:40:27.259480 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.259458    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/944a456c-d53c-434a-850c-b7418f7f6116-kserve-provision-location\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:40:27.259537 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.259492    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/944a456c-d53c-434a-850c-b7418f7f6116-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "944a456c-d53c-434a-850c-b7418f7f6116" (UID: "944a456c-d53c-434a-850c-b7418f7f6116"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:40:27.259743 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.259721    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/944a456c-d53c-434a-850c-b7418f7f6116-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "944a456c-d53c-434a-850c-b7418f7f6116" (UID: "944a456c-d53c-434a-850c-b7418f7f6116"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:40:27.261472 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.261452    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/944a456c-d53c-434a-850c-b7418f7f6116-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "944a456c-d53c-434a-850c-b7418f7f6116" (UID: "944a456c-d53c-434a-850c-b7418f7f6116"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:40:27.261674 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.261658    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/944a456c-d53c-434a-850c-b7418f7f6116-kube-api-access-lv5v5" (OuterVolumeSpecName: "kube-api-access-lv5v5") pod "944a456c-d53c-434a-850c-b7418f7f6116" (UID: "944a456c-d53c-434a-850c-b7418f7f6116"). InnerVolumeSpecName "kube-api-access-lv5v5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:40:27.264432 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.264409    2581 scope.go:117] "RemoveContainer" containerID="f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796"
Jul 01 19:40:27.273371 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.273352    2581 scope.go:117] "RemoveContainer" containerID="8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3"
Jul 01 19:40:27.281764 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.281746    2581 scope.go:117] "RemoveContainer" containerID="9dc17f5b35156bd51de91f8508a35faad8c2b7d8e5f9fb1ba93a133ac24d8668"
Jul 01 19:40:27.289657 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.289632    2581 scope.go:117] "RemoveContainer" containerID="eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9"
Jul 01 19:40:27.289969 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:40:27.289949    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9\": container with ID starting with eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9 not found: ID does not exist" containerID="eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9"
Jul 01 19:40:27.290043 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.289990    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9"} err="failed to get container status \"eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9\": rpc error: code = NotFound desc = could not find container \"eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9\": container with ID starting with eb7a74ce8c845334700efab42e450b4aa9c324b96c76654e27e4feb6febe24c9 not found: ID does not exist"
Jul 01 19:40:27.290043 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.290012    2581 scope.go:117] "RemoveContainer" containerID="f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796"
Jul 01 19:40:27.290276 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:40:27.290256    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796\": container with ID starting with f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796 not found: ID does not exist" containerID="f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796"
Jul 01 19:40:27.290331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.290285    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796"} err="failed to get container status \"f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796\": rpc error: code = NotFound desc = could not find container \"f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796\": container with ID starting with f14ac31169cd63ce0770569578bfd39abebc61e39eca2926356244fa60e7f796 not found: ID does not exist"
Jul 01 19:40:27.290331 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.290301    2581 scope.go:117] "RemoveContainer" containerID="8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3"
Jul 01 19:40:27.290590 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:40:27.290554    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3\": container with ID starting with 8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3 not found: ID does not exist" containerID="8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3"
Jul 01 19:40:27.290648 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.290595    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3"} err="failed to get container status \"8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3\": rpc error: code = NotFound desc = could not find container \"8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3\": container with ID starting with 8864ee2bd700a4e65953ccf7791f2a0357844facd5a99fc5a2c3bdcc013454c3 not found: ID does not exist"
Jul 01 19:40:27.290648 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.290619    2581 scope.go:117] "RemoveContainer" containerID="9dc17f5b35156bd51de91f8508a35faad8c2b7d8e5f9fb1ba93a133ac24d8668"
Jul 01 19:40:27.290882 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:40:27.290864    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dc17f5b35156bd51de91f8508a35faad8c2b7d8e5f9fb1ba93a133ac24d8668\": container with ID starting with 9dc17f5b35156bd51de91f8508a35faad8c2b7d8e5f9fb1ba93a133ac24d8668 not found: ID does not exist" containerID="9dc17f5b35156bd51de91f8508a35faad8c2b7d8e5f9fb1ba93a133ac24d8668"
Jul 01 19:40:27.290941 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.290888    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc17f5b35156bd51de91f8508a35faad8c2b7d8e5f9fb1ba93a133ac24d8668"} err="failed to get container status \"9dc17f5b35156bd51de91f8508a35faad8c2b7d8e5f9fb1ba93a133ac24d8668\": rpc error: code = NotFound desc = could not find container \"9dc17f5b35156bd51de91f8508a35faad8c2b7d8e5f9fb1ba93a133ac24d8668\": container with ID starting with 9dc17f5b35156bd51de91f8508a35faad8c2b7d8e5f9fb1ba93a133ac24d8668 not found: ID does not exist"
Jul 01 19:40:27.360788 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.360749    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/944a456c-d53c-434a-850c-b7418f7f6116-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:40:27.360788 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.360781    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/944a456c-d53c-434a-850c-b7418f7f6116-proxy-tls\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:40:27.361013 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.360797    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lv5v5\" (UniqueName: \"kubernetes.io/projected/944a456c-d53c-434a-850c-b7418f7f6116-kube-api-access-lv5v5\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:40:27.361013 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.360813    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/944a456c-d53c-434a-850c-b7418f7f6116-cabundle-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:40:27.578406 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.578371    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"]
Jul 01 19:40:27.579989 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:27.579967    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5597c66d9b-tqs97"]
Jul 01 19:40:29.221664 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:40:29.221628    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="944a456c-d53c-434a-850c-b7418f7f6116" path="/var/lib/kubelet/pods/944a456c-d53c-434a-850c-b7418f7f6116/volumes"
Jul 01 19:41:29.231635 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:41:29.231601    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:41:29.235541 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:41:29.235517    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:42:45.995926 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:42:45.995898    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" event={"ID":"d989ba85-7454-4631-98f2-3d8dcd50aeeb","Type":"ContainerStarted","Data":"ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8"}
Jul 01 19:42:47.002624 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:42:47.002588    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" event={"ID":"d989ba85-7454-4631-98f2-3d8dcd50aeeb","Type":"ContainerStarted","Data":"71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674"}
Jul 01 19:42:47.003210 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:42:47.002771    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:42:47.003210 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:42:47.002937    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:42:47.004004 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:42:47.003975    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 01 19:42:47.040797 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:42:47.040752    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" podStartSLOduration=3.582796789 podStartE2EDuration="2m51.040738212s" totalImagesPullingTime="2m18.598635609s" totalInitContainerRuntime="28.859305814s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:39:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:42:47.037428218 +0000 UTC m=+1278.272553542" watchObservedRunningTime="2026-07-01 19:42:47.040738212 +0000 UTC m=+1278.275863525"
Jul 01 19:42:48.007160 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:42:48.007121    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 01 19:42:53.011759 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:42:53.011732    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:42:53.012304 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:42:53.012273    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 01 19:43:03.012791 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:03.012760    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:43:07.121822 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:07.121786    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"]
Jul 01 19:43:07.122247 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:07.122014    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="kserve-container" containerID="cri-o://ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8" gracePeriod=30
Jul 01 19:43:07.122247 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:07.122082    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="kube-rbac-proxy" containerID="cri-o://71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674" gracePeriod=30
Jul 01 19:43:08.106647 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.106547    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:43:08.117414 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.117382    2581 generic.go:356] "Generic (PLEG): container finished" podID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerID="71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674" exitCode=2
Jul 01 19:43:08.117414 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.117409    2581 generic.go:356] "Generic (PLEG): container finished" podID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerID="ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8" exitCode=0
Jul 01 19:43:08.117637 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.117447    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"
Jul 01 19:43:08.117637 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.117475    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" event={"ID":"d989ba85-7454-4631-98f2-3d8dcd50aeeb","Type":"ContainerDied","Data":"71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674"}
Jul 01 19:43:08.117637 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.117594    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" event={"ID":"d989ba85-7454-4631-98f2-3d8dcd50aeeb","Type":"ContainerDied","Data":"ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8"}
Jul 01 19:43:08.117637 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.117627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" event={"ID":"d989ba85-7454-4631-98f2-3d8dcd50aeeb","Type":"ContainerDied","Data":"3c1549e0802351bb2907c39ec961c11cab52f7b0c2a576a0580d1f31616b5c04"}
Jul 01 19:43:08.117779 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.117661    2581 scope.go:117] "RemoveContainer" containerID="71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674"
Jul 01 19:43:08.126104 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.125891    2581 scope.go:117] "RemoveContainer" containerID="ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8"
Jul 01 19:43:08.134917 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.134899    2581 scope.go:117] "RemoveContainer" containerID="7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4"
Jul 01 19:43:08.142975 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.142950    2581 scope.go:117] "RemoveContainer" containerID="71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674"
Jul 01 19:43:08.143189 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:43:08.143171    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674\": container with ID starting with 71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674 not found: ID does not exist" containerID="71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674"
Jul 01 19:43:08.143241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.143196    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674"} err="failed to get container status \"71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674\": rpc error: code = NotFound desc = could not find container \"71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674\": container with ID starting with 71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674 not found: ID does not exist"
Jul 01 19:43:08.143241 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.143213    2581 scope.go:117] "RemoveContainer" containerID="ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8"
Jul 01 19:43:08.143444 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:43:08.143428    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8\": container with ID starting with ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8 not found: ID does not exist" containerID="ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8"
Jul 01 19:43:08.143493 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.143462    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8"} err="failed to get container status \"ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8\": rpc error: code = NotFound desc = could not find container \"ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8\": container with ID starting with ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8 not found: ID does not exist"
Jul 01 19:43:08.143493 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.143480    2581 scope.go:117] "RemoveContainer" containerID="7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4"
Jul 01 19:43:08.143735 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:43:08.143714    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4\": container with ID starting with 7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4 not found: ID does not exist" containerID="7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4"
Jul 01 19:43:08.143820 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.143739    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4"} err="failed to get container status \"7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4\": rpc error: code = NotFound desc = could not find container \"7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4\": container with ID starting with 7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4 not found: ID does not exist"
Jul 01 19:43:08.143820 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.143757    2581 scope.go:117] "RemoveContainer" containerID="71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674"
Jul 01 19:43:08.143960 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.143941    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674"} err="failed to get container status \"71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674\": rpc error: code = NotFound desc = could not find container \"71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674\": container with ID starting with 71776674246b9c44f5257fcfbbdd7dc9fde6ead2046ff6a442d1f3d516407674 not found: ID does not exist"
Jul 01 19:43:08.144032 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.143963    2581 scope.go:117] "RemoveContainer" containerID="ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8"
Jul 01 19:43:08.144212 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.144195    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8"} err="failed to get container status \"ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8\": rpc error: code = NotFound desc = could not find container \"ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8\": container with ID starting with ec7cffdc049871c8902cb64e8f1e8d22e60b88d9bee2cca5b584845753381de8 not found: ID does not exist"
Jul 01 19:43:08.144257 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.144213    2581 scope.go:117] "RemoveContainer" containerID="7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4"
Jul 01 19:43:08.144420 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.144401    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4"} err="failed to get container status \"7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4\": rpc error: code = NotFound desc = could not find container \"7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4\": container with ID starting with 7c109b4e2d1725006741eeb5a5e40b4402b65550b7d9825bca9886e132f967b4 not found: ID does not exist"
Jul 01 19:43:08.183941 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.183901    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d989ba85-7454-4631-98f2-3d8dcd50aeeb-cabundle-cert\") pod \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") "
Jul 01 19:43:08.184176 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.183965    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kube-rbac-proxy-sar-config\") pod \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") "
Jul 01 19:43:08.184176 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.184027    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lcf5\" (UniqueName: \"kubernetes.io/projected/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kube-api-access-7lcf5\") pod \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") "
Jul 01 19:43:08.184176 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.184052    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kserve-provision-location\") pod \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") "
Jul 01 19:43:08.184176 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.184117    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d989ba85-7454-4631-98f2-3d8dcd50aeeb-proxy-tls\") pod \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\" (UID: \"d989ba85-7454-4631-98f2-3d8dcd50aeeb\") "
Jul 01 19:43:08.184382 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.184346    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d989ba85-7454-4631-98f2-3d8dcd50aeeb-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "d989ba85-7454-4631-98f2-3d8dcd50aeeb" (UID: "d989ba85-7454-4631-98f2-3d8dcd50aeeb"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:43:08.184382 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.184356    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d989ba85-7454-4631-98f2-3d8dcd50aeeb" (UID: "d989ba85-7454-4631-98f2-3d8dcd50aeeb"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:43:08.186190 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.186166    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kube-api-access-7lcf5" (OuterVolumeSpecName: "kube-api-access-7lcf5") pod "d989ba85-7454-4631-98f2-3d8dcd50aeeb" (UID: "d989ba85-7454-4631-98f2-3d8dcd50aeeb"). InnerVolumeSpecName "kube-api-access-7lcf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:43:08.186315 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.186164    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d989ba85-7454-4631-98f2-3d8dcd50aeeb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d989ba85-7454-4631-98f2-3d8dcd50aeeb" (UID: "d989ba85-7454-4631-98f2-3d8dcd50aeeb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:43:08.188901 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.188875    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d989ba85-7454-4631-98f2-3d8dcd50aeeb" (UID: "d989ba85-7454-4631-98f2-3d8dcd50aeeb"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:43:08.285539 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.285503    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d989ba85-7454-4631-98f2-3d8dcd50aeeb-cabundle-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:43:08.285737 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.285601    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:43:08.285737 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.285612    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lcf5\" (UniqueName: \"kubernetes.io/projected/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kube-api-access-7lcf5\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:43:08.285737 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.285621    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d989ba85-7454-4631-98f2-3d8dcd50aeeb-kserve-provision-location\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:43:08.285737 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.285631    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d989ba85-7454-4631-98f2-3d8dcd50aeeb-proxy-tls\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:43:08.440293 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.440262    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"]
Jul 01 19:43:08.443675 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:08.443651    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr"]
Jul 01 19:43:09.008035 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:09.007984    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-vscfr" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.38:8643/healthz\": context deadline exceeded"
Jul 01 19:43:09.215995 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:09.215960    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" path="/var/lib/kubelet/pods/d989ba85-7454-4631-98f2-3d8dcd50aeeb/volumes"
Jul 01 19:43:12.418171 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418140    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"]
Jul 01 19:43:12.418591 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418506    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kube-rbac-proxy"
Jul 01 19:43:12.418591 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418518    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kube-rbac-proxy"
Jul 01 19:43:12.418591 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418532    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="kserve-container"
Jul 01 19:43:12.418591 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418537    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="kserve-container"
Jul 01 19:43:12.418591 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418550    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container"
Jul 01 19:43:12.418591 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418556    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container"
Jul 01 19:43:12.418591 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418562    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent"
Jul 01 19:43:12.418591 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418567    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent"
Jul 01 19:43:12.418591 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418586    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="kube-rbac-proxy"
Jul 01 19:43:12.418591 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418592    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="kube-rbac-proxy"
Jul 01 19:43:12.418906 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418608    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="storage-initializer"
Jul 01 19:43:12.418906 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418613    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="storage-initializer"
Jul 01 19:43:12.418906 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418624    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="storage-initializer"
Jul 01 19:43:12.418906 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418629    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="storage-initializer"
Jul 01 19:43:12.418906 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418681    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="agent"
Jul 01 19:43:12.418906 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418691    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kube-rbac-proxy"
Jul 01 19:43:12.418906 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418698    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="kube-rbac-proxy"
Jul 01 19:43:12.418906 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418704    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="944a456c-d53c-434a-850c-b7418f7f6116" containerName="kserve-container"
Jul 01 19:43:12.418906 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.418709    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d989ba85-7454-4631-98f2-3d8dcd50aeeb" containerName="kserve-container"
Jul 01 19:43:12.446218 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.446173    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"]
Jul 01 19:43:12.446381 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.446297    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.449496 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.449473    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-kube-rbac-proxy-sar-config\""
Jul 01 19:43:12.449496 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.449492    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 19:43:12.449743 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.449615    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 19:43:12.450961 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.450942    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2q28w\""
Jul 01 19:43:12.451071 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.450980    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 19:43:12.451071 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.450944    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 19:43:12.451071 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.450988    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-predictor-serving-cert\""
Jul 01 19:43:12.527334 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.527297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.527334 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.527335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gnqs\" (UniqueName: \"kubernetes.io/projected/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kube-api-access-5gnqs\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.527551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.527367    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1fa541ef-6ca8-4786-97b8-111b8ff72baa-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.527551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.527430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1fa541ef-6ca8-4786-97b8-111b8ff72baa-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.527551 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.527465    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.628631 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.628567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1fa541ef-6ca8-4786-97b8-111b8ff72baa-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.628830 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.628645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.628830 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.628708    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.628830 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.628758    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gnqs\" (UniqueName: \"kubernetes.io/projected/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kube-api-access-5gnqs\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.628830 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.628794    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1fa541ef-6ca8-4786-97b8-111b8ff72baa-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.629085 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.629058    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.629295 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.629273    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1fa541ef-6ca8-4786-97b8-111b8ff72baa-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.629354 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.629317    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.631168 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.631146    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1fa541ef-6ca8-4786-97b8-111b8ff72baa-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.637151 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.637127    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gnqs\" (UniqueName: \"kubernetes.io/projected/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kube-api-access-5gnqs\") pod \"isvc-autogluon-v2-predictor-67b849d456-pnjwc\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.757462 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.757435    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:12.905036 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:12.905006    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"]
Jul 01 19:43:12.906850 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:43:12.906825    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa541ef_6ca8_4786_97b8_111b8ff72baa.slice/crio-ccc6ba47d67505ac13ed5ea023aea49b81e16bc6070e6742fab4103c7300c6b7 WatchSource:0}: Error finding container ccc6ba47d67505ac13ed5ea023aea49b81e16bc6070e6742fab4103c7300c6b7: Status 404 returned error can't find the container with id ccc6ba47d67505ac13ed5ea023aea49b81e16bc6070e6742fab4103c7300c6b7
Jul 01 19:43:13.143214 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:13.143173    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc" event={"ID":"1fa541ef-6ca8-4786-97b8-111b8ff72baa","Type":"ContainerStarted","Data":"5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b"}
Jul 01 19:43:13.143413 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:13.143239    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc" event={"ID":"1fa541ef-6ca8-4786-97b8-111b8ff72baa","Type":"ContainerStarted","Data":"ccc6ba47d67505ac13ed5ea023aea49b81e16bc6070e6742fab4103c7300c6b7"}
Jul 01 19:43:18.170372 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:18.170337    2581 generic.go:356] "Generic (PLEG): container finished" podID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerID="5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b" exitCode=0
Jul 01 19:43:18.170919 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:18.170421    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc" event={"ID":"1fa541ef-6ca8-4786-97b8-111b8ff72baa","Type":"ContainerDied","Data":"5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b"}
Jul 01 19:43:19.178998 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:19.178960    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc" event={"ID":"1fa541ef-6ca8-4786-97b8-111b8ff72baa","Type":"ContainerStarted","Data":"6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285"}
Jul 01 19:43:19.179482 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:19.179058    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc" event={"ID":"1fa541ef-6ca8-4786-97b8-111b8ff72baa","Type":"ContainerStarted","Data":"1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e"}
Jul 01 19:43:19.179482 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:19.179250    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:19.208417 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:19.208351    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc" podStartSLOduration=2.6606718750000002 podStartE2EDuration="7.208334951s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.547663076s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:43:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:43:19.205495303 +0000 UTC m=+1310.440620619" watchObservedRunningTime="2026-07-01 19:43:19.208334951 +0000 UTC m=+1310.443460263"
Jul 01 19:43:20.183845 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:20.183816    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:43:26.194256 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:43:26.194223    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:44:55.215796 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:44:55.215768    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:45:02.577534 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:02.577502    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"]
Jul 01 19:45:02.577992 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:02.577798    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc" podUID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerName="kserve-container" containerID="cri-o://1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e" gracePeriod=30
Jul 01 19:45:02.577992 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:02.577860    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc" podUID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerName="kube-rbac-proxy" containerID="cri-o://6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285" gracePeriod=30
Jul 01 19:45:03.528323 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.528298    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:45:03.603503 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.603404    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kserve-provision-location\") pod \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") "
Jul 01 19:45:03.603503 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.603486    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gnqs\" (UniqueName: \"kubernetes.io/projected/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kube-api-access-5gnqs\") pod \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") "
Jul 01 19:45:03.604087 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.603508    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1fa541ef-6ca8-4786-97b8-111b8ff72baa-cabundle-cert\") pod \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") "
Jul 01 19:45:03.604087 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.603533    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1fa541ef-6ca8-4786-97b8-111b8ff72baa-proxy-tls\") pod \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") "
Jul 01 19:45:03.604087 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.603566    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kube-rbac-proxy-sar-config\") pod \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\" (UID: \"1fa541ef-6ca8-4786-97b8-111b8ff72baa\") "
Jul 01 19:45:03.604087 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.603962    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fa541ef-6ca8-4786-97b8-111b8ff72baa-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "1fa541ef-6ca8-4786-97b8-111b8ff72baa" (UID: "1fa541ef-6ca8-4786-97b8-111b8ff72baa"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:45:03.604087 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.604036    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1fa541ef-6ca8-4786-97b8-111b8ff72baa" (UID: "1fa541ef-6ca8-4786-97b8-111b8ff72baa"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:45:03.605647 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.605626    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kube-api-access-5gnqs" (OuterVolumeSpecName: "kube-api-access-5gnqs") pod "1fa541ef-6ca8-4786-97b8-111b8ff72baa" (UID: "1fa541ef-6ca8-4786-97b8-111b8ff72baa"). InnerVolumeSpecName "kube-api-access-5gnqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:45:03.605754 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.605664    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fa541ef-6ca8-4786-97b8-111b8ff72baa-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1fa541ef-6ca8-4786-97b8-111b8ff72baa" (UID: "1fa541ef-6ca8-4786-97b8-111b8ff72baa"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:45:03.609845 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.609816    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1fa541ef-6ca8-4786-97b8-111b8ff72baa" (UID: "1fa541ef-6ca8-4786-97b8-111b8ff72baa"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:45:03.670032 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.669996    2581 generic.go:356] "Generic (PLEG): container finished" podID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerID="6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285" exitCode=2
Jul 01 19:45:03.670032 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.670023    2581 generic.go:356] "Generic (PLEG): container finished" podID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerID="1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e" exitCode=0
Jul 01 19:45:03.670236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.670056    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"
Jul 01 19:45:03.670236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.670078    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc" event={"ID":"1fa541ef-6ca8-4786-97b8-111b8ff72baa","Type":"ContainerDied","Data":"6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285"}
Jul 01 19:45:03.670236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.670144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc" event={"ID":"1fa541ef-6ca8-4786-97b8-111b8ff72baa","Type":"ContainerDied","Data":"1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e"}
Jul 01 19:45:03.670236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.670179    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc" event={"ID":"1fa541ef-6ca8-4786-97b8-111b8ff72baa","Type":"ContainerDied","Data":"ccc6ba47d67505ac13ed5ea023aea49b81e16bc6070e6742fab4103c7300c6b7"}
Jul 01 19:45:03.670236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.670207    2581 scope.go:117] "RemoveContainer" containerID="6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285"
Jul 01 19:45:03.679075 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.679057    2581 scope.go:117] "RemoveContainer" containerID="1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e"
Jul 01 19:45:03.686689 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.686666    2581 scope.go:117] "RemoveContainer" containerID="5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b"
Jul 01 19:45:03.694517 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.694495    2581 scope.go:117] "RemoveContainer" containerID="6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285"
Jul 01 19:45:03.694850 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:45:03.694829    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285\": container with ID starting with 6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285 not found: ID does not exist" containerID="6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285"
Jul 01 19:45:03.694924 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.694858    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285"} err="failed to get container status \"6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285\": rpc error: code = NotFound desc = could not find container \"6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285\": container with ID starting with 6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285 not found: ID does not exist"
Jul 01 19:45:03.694924 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.694876    2581 scope.go:117] "RemoveContainer" containerID="1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e"
Jul 01 19:45:03.695161 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:45:03.695140    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e\": container with ID starting with 1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e not found: ID does not exist" containerID="1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e"
Jul 01 19:45:03.695213 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.695167    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e"} err="failed to get container status \"1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e\": rpc error: code = NotFound desc = could not find container \"1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e\": container with ID starting with 1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e not found: ID does not exist"
Jul 01 19:45:03.695213 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.695186    2581 scope.go:117] "RemoveContainer" containerID="5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b"
Jul 01 19:45:03.695472 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:45:03.695456    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b\": container with ID starting with 5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b not found: ID does not exist" containerID="5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b"
Jul 01 19:45:03.695513 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.695476    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b"} err="failed to get container status \"5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b\": rpc error: code = NotFound desc = could not find container \"5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b\": container with ID starting with 5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b not found: ID does not exist"
Jul 01 19:45:03.695513 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.695489    2581 scope.go:117] "RemoveContainer" containerID="6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285"
Jul 01 19:45:03.695759 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.695737    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285"} err="failed to get container status \"6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285\": rpc error: code = NotFound desc = could not find container \"6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285\": container with ID starting with 6975eb44bd35cee8f45add1ea8d7c997abc0b48f1668011f198c2d10b17c5285 not found: ID does not exist"
Jul 01 19:45:03.695837 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.695762    2581 scope.go:117] "RemoveContainer" containerID="1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e"
Jul 01 19:45:03.696029 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.696008    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e"} err="failed to get container status \"1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e\": rpc error: code = NotFound desc = could not find container \"1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e\": container with ID starting with 1409d705459d8201bc193452dafe25baaaa21c1cb5fae1291ead10a9fe52451e not found: ID does not exist"
Jul 01 19:45:03.696085 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.696030    2581 scope.go:117] "RemoveContainer" containerID="5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b"
Jul 01 19:45:03.696301 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.696279    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b"} err="failed to get container status \"5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b\": rpc error: code = NotFound desc = could not find container \"5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b\": container with ID starting with 5e4c3b4c70754ba81b57e318186d6b41714e03f2057b02a5a5ba5d0a36a6cb3b not found: ID does not exist"
Jul 01 19:45:03.696374 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.696359    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"]
Jul 01 19:45:03.701936 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.701915    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pnjwc"]
Jul 01 19:45:03.704521 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.704503    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gnqs\" (UniqueName: \"kubernetes.io/projected/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kube-api-access-5gnqs\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:45:03.704628 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.704523    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1fa541ef-6ca8-4786-97b8-111b8ff72baa-cabundle-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:45:03.704628 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.704534    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1fa541ef-6ca8-4786-97b8-111b8ff72baa-proxy-tls\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:45:03.704628 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.704542    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:45:03.704628 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:03.704550    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1fa541ef-6ca8-4786-97b8-111b8ff72baa-kserve-provision-location\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:45:05.215208 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:05.215171    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" path="/var/lib/kubelet/pods/1fa541ef-6ca8-4786-97b8-111b8ff72baa/volumes"
Jul 01 19:45:07.744849 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.744816    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"]
Jul 01 19:45:07.745236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.745158    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerName="kserve-container"
Jul 01 19:45:07.745236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.745168    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerName="kserve-container"
Jul 01 19:45:07.745236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.745187    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerName="storage-initializer"
Jul 01 19:45:07.745236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.745192    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerName="storage-initializer"
Jul 01 19:45:07.745236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.745207    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerName="kube-rbac-proxy"
Jul 01 19:45:07.745236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.745212    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerName="kube-rbac-proxy"
Jul 01 19:45:07.745419 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.745292    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerName="kserve-container"
Jul 01 19:45:07.745419 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.745310    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fa541ef-6ca8-4786-97b8-111b8ff72baa" containerName="kube-rbac-proxy"
Jul 01 19:45:07.748983 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.748964    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.751927 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.751902    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2q28w\""
Jul 01 19:45:07.752090 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.752072    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 19:45:07.752090 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.752086    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-predictor-serving-cert\""
Jul 01 19:45:07.753358 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.753344    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-kube-rbac-proxy-sar-config\""
Jul 01 19:45:07.753455 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.753374    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 19:45:07.753455 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.753347    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 19:45:07.753556 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.753453    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 19:45:07.760023 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.760002    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"]
Jul 01 19:45:07.839209 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.839169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqfts\" (UniqueName: \"kubernetes.io/projected/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kube-api-access-qqfts\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.839377 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.839253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.839377 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.839288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.839377 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.839322    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.839487 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.839381    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.939879 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.939837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqfts\" (UniqueName: \"kubernetes.io/projected/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kube-api-access-qqfts\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.940057 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.939918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.940057 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.939937    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.940057 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.939968    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.940057 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.939987    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.940222 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:45:07.940082    2581 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-serving-cert: secret "isvc-autogluon-v2-variants-predictor-serving-cert" not found
Jul 01 19:45:07.940222 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:45:07.940152    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-proxy-tls podName:87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f nodeName:}" failed. No retries permitted until 2026-07-01 19:45:08.440131701 +0000 UTC m=+1419.675256991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-proxy-tls") pod "isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5" (UID: "87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f") : secret "isvc-autogluon-v2-variants-predictor-serving-cert" not found
Jul 01 19:45:07.940365 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.940345    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.940657 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.940638    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.940731 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.940716    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:07.950053 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:07.950030    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqfts\" (UniqueName: \"kubernetes.io/projected/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kube-api-access-qqfts\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:08.444504 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:08.444462    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:08.446920 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:08.446899    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:08.660243 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:08.660213    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:08.797161 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:08.797135    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"]
Jul 01 19:45:08.799497 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:45:08.799460    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87d17a6b_1f1c_4ba9_83bc_a2daf6f0a55f.slice/crio-ac37876e94fdf628b2238ba2b3ad176460e069819337171215d2d5edc96c241c WatchSource:0}: Error finding container ac37876e94fdf628b2238ba2b3ad176460e069819337171215d2d5edc96c241c: Status 404 returned error can't find the container with id ac37876e94fdf628b2238ba2b3ad176460e069819337171215d2d5edc96c241c
Jul 01 19:45:09.701688 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:09.701646    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5" event={"ID":"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f","Type":"ContainerStarted","Data":"3bb1c38e52921e5e4474f2c71f781a7eb8c325da07c15242674654a8739b8838"}
Jul 01 19:45:09.701852 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:09.701713    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5" event={"ID":"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f","Type":"ContainerStarted","Data":"ac37876e94fdf628b2238ba2b3ad176460e069819337171215d2d5edc96c241c"}
Jul 01 19:45:14.727399 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:14.727365    2581 generic.go:356] "Generic (PLEG): container finished" podID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerID="3bb1c38e52921e5e4474f2c71f781a7eb8c325da07c15242674654a8739b8838" exitCode=0
Jul 01 19:45:14.727808 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:14.727439    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5" event={"ID":"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f","Type":"ContainerDied","Data":"3bb1c38e52921e5e4474f2c71f781a7eb8c325da07c15242674654a8739b8838"}
Jul 01 19:45:15.735706 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:15.735673    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5" event={"ID":"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f","Type":"ContainerStarted","Data":"97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7"}
Jul 01 19:45:15.736096 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:15.735731    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5" event={"ID":"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f","Type":"ContainerStarted","Data":"4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186"}
Jul 01 19:45:15.736096 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:15.736013    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:15.736096 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:15.736068    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:45:15.764126 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:15.764064    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5" podStartSLOduration=3.723236659 podStartE2EDuration="8.764047746s" totalImagesPullingTime="0s" totalInitContainerRuntime="5.040811087s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:45:15.761805039 +0000 UTC m=+1426.996930352" watchObservedRunningTime="2026-07-01 19:45:15.764047746 +0000 UTC m=+1426.999173107"
Jul 01 19:45:21.748920 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:45:21.748827    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:46:29.261756 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:29.261727    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:46:29.266651 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:29.266629    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:46:52.213910 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:52.213879    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:46:57.929491 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:57.929455    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"]
Jul 01 19:46:57.930052 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:57.929705    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5" podUID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerName="kserve-container" containerID="cri-o://4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186" gracePeriod=30
Jul 01 19:46:57.930052 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:57.929756    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5" podUID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerName="kube-rbac-proxy" containerID="cri-o://97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7" gracePeriod=30
Jul 01 19:46:58.222363 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:58.222326    2581 generic.go:356] "Generic (PLEG): container finished" podID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerID="97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7" exitCode=2
Jul 01 19:46:58.222633 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:58.222408    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5" event={"ID":"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f","Type":"ContainerDied","Data":"97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7"}
Jul 01 19:46:58.986127 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:58.986101    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:46:59.127661 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.127532    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kube-rbac-proxy-sar-config\") pod \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") "
Jul 01 19:46:59.127661 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.127625    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-cabundle-cert\") pod \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") "
Jul 01 19:46:59.127873 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.127676    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-proxy-tls\") pod \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") "
Jul 01 19:46:59.127873 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.127731    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqfts\" (UniqueName: \"kubernetes.io/projected/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kube-api-access-qqfts\") pod \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") "
Jul 01 19:46:59.127873 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.127754    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kserve-provision-location\") pod \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\" (UID: \"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f\") "
Jul 01 19:46:59.128003 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.127971    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" (UID: "87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:46:59.128162 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.128132    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" (UID: "87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:46:59.129827 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.129797    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kube-api-access-qqfts" (OuterVolumeSpecName: "kube-api-access-qqfts") pod "87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" (UID: "87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f"). InnerVolumeSpecName "kube-api-access-qqfts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:46:59.129827 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.129809    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" (UID: "87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:46:59.134431 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.134398    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" (UID: "87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:46:59.228454 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.228421    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqfts\" (UniqueName: \"kubernetes.io/projected/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kube-api-access-qqfts\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:46:59.228454 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.228452    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kserve-provision-location\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:46:59.228718 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.228469    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:46:59.228718 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.228483    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-cabundle-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:46:59.228718 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.228497    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f-proxy-tls\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:46:59.229993 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.229953    2581 generic.go:356] "Generic (PLEG): container finished" podID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerID="4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186" exitCode=0
Jul 01 19:46:59.230137 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.230054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5" event={"ID":"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f","Type":"ContainerDied","Data":"4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186"}
Jul 01 19:46:59.230337 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.230073    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"
Jul 01 19:46:59.230337 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.230170    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5" event={"ID":"87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f","Type":"ContainerDied","Data":"ac37876e94fdf628b2238ba2b3ad176460e069819337171215d2d5edc96c241c"}
Jul 01 19:46:59.230337 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.230221    2581 scope.go:117] "RemoveContainer" containerID="97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7"
Jul 01 19:46:59.239644 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.239623    2581 scope.go:117] "RemoveContainer" containerID="4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186"
Jul 01 19:46:59.247905 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.247883    2581 scope.go:117] "RemoveContainer" containerID="3bb1c38e52921e5e4474f2c71f781a7eb8c325da07c15242674654a8739b8838"
Jul 01 19:46:59.251236 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.251208    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"]
Jul 01 19:46:59.257308 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.257286    2581 scope.go:117] "RemoveContainer" containerID="97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7"
Jul 01 19:46:59.257486 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.257466    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lrxc5"]
Jul 01 19:46:59.257645 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:46:59.257624    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7\": container with ID starting with 97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7 not found: ID does not exist" containerID="97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7"
Jul 01 19:46:59.257699 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.257653    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7"} err="failed to get container status \"97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7\": rpc error: code = NotFound desc = could not find container \"97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7\": container with ID starting with 97a11851f4402bd8dc00620bd5d709466c93cc332de50b44bfcaec2e9c6ef5f7 not found: ID does not exist"
Jul 01 19:46:59.257699 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.257673    2581 scope.go:117] "RemoveContainer" containerID="4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186"
Jul 01 19:46:59.257938 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:46:59.257920    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186\": container with ID starting with 4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186 not found: ID does not exist" containerID="4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186"
Jul 01 19:46:59.257981 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.257947    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186"} err="failed to get container status \"4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186\": rpc error: code = NotFound desc = could not find container \"4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186\": container with ID starting with 4244d12a20a31174c107bd21d956f6bef75dbd1bfe2ba4a42edd3011fb4c5186 not found: ID does not exist"
Jul 01 19:46:59.257981 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.257965    2581 scope.go:117] "RemoveContainer" containerID="3bb1c38e52921e5e4474f2c71f781a7eb8c325da07c15242674654a8739b8838"
Jul 01 19:46:59.258228 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:46:59.258212    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bb1c38e52921e5e4474f2c71f781a7eb8c325da07c15242674654a8739b8838\": container with ID starting with 3bb1c38e52921e5e4474f2c71f781a7eb8c325da07c15242674654a8739b8838 not found: ID does not exist" containerID="3bb1c38e52921e5e4474f2c71f781a7eb8c325da07c15242674654a8739b8838"
Jul 01 19:46:59.258272 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:46:59.258234    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb1c38e52921e5e4474f2c71f781a7eb8c325da07c15242674654a8739b8838"} err="failed to get container status \"3bb1c38e52921e5e4474f2c71f781a7eb8c325da07c15242674654a8739b8838\": rpc error: code = NotFound desc = could not find container \"3bb1c38e52921e5e4474f2c71f781a7eb8c325da07c15242674654a8739b8838\": container with ID starting with 3bb1c38e52921e5e4474f2c71f781a7eb8c325da07c15242674654a8739b8838 not found: ID does not exist"
Jul 01 19:47:01.216249 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:01.216214    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" path="/var/lib/kubelet/pods/87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f/volumes"
Jul 01 19:47:03.023359 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.023317    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"]
Jul 01 19:47:03.023884 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.023861    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerName="kube-rbac-proxy"
Jul 01 19:47:03.023884 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.023881    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerName="kube-rbac-proxy"
Jul 01 19:47:03.024007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.023898    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerName="storage-initializer"
Jul 01 19:47:03.024007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.023908    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerName="storage-initializer"
Jul 01 19:47:03.024007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.023925    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerName="kserve-container"
Jul 01 19:47:03.024007 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.023934    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerName="kserve-container"
Jul 01 19:47:03.024207 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.024048    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerName="kserve-container"
Jul 01 19:47:03.024207 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.024065    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="87d17a6b-1f1c-4ba9-83bc-a2daf6f0a55f" containerName="kube-rbac-proxy"
Jul 01 19:47:03.027473 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.027450    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.030823 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.030803    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 19:47:03.031299 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.031281    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 19:47:03.031412 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.031393    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-predictor-serving-cert\""
Jul 01 19:47:03.031493 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.031474    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 19:47:03.031546 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.031474    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-kube-rbac-proxy-sar-config\""
Jul 01 19:47:03.031754 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.031739    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 19:47:03.032371 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.032351    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2q28w\""
Jul 01 19:47:03.039716 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.039691    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"]
Jul 01 19:47:03.164412 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.164372    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3e882572-826f-4b10-9579-e2df0af9cf21-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.164631 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.164422    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59jsw\" (UniqueName: \"kubernetes.io/projected/3e882572-826f-4b10-9579-e2df0af9cf21-kube-api-access-59jsw\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.164631 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.164468    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3e882572-826f-4b10-9579-e2df0af9cf21-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.164631 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.164490    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3e882572-826f-4b10-9579-e2df0af9cf21-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.164631 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.164510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3e882572-826f-4b10-9579-e2df0af9cf21-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.265799 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.265759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3e882572-826f-4b10-9579-e2df0af9cf21-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.265987 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.265810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59jsw\" (UniqueName: \"kubernetes.io/projected/3e882572-826f-4b10-9579-e2df0af9cf21-kube-api-access-59jsw\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.265987 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.265835    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3e882572-826f-4b10-9579-e2df0af9cf21-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.265987 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.265852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3e882572-826f-4b10-9579-e2df0af9cf21-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.265987 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.265876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3e882572-826f-4b10-9579-e2df0af9cf21-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.266313 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.266291    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3e882572-826f-4b10-9579-e2df0af9cf21-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.266602 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.266552    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3e882572-826f-4b10-9579-e2df0af9cf21-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.266602 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.266595    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3e882572-826f-4b10-9579-e2df0af9cf21-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.268270 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.268251    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3e882572-826f-4b10-9579-e2df0af9cf21-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.275808 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.275751    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59jsw\" (UniqueName: \"kubernetes.io/projected/3e882572-826f-4b10-9579-e2df0af9cf21-kube-api-access-59jsw\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.338707 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.338665    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:03.478814 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:03.478788    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"]
Jul 01 19:47:03.480905 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:47:03.480867    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e882572_826f_4b10_9579_e2df0af9cf21.slice/crio-c72451e56b70ca940b8a5e971de8b55a5b89ef01127aaab968e34f6567e4f2f4 WatchSource:0}: Error finding container c72451e56b70ca940b8a5e971de8b55a5b89ef01127aaab968e34f6567e4f2f4: Status 404 returned error can't find the container with id c72451e56b70ca940b8a5e971de8b55a5b89ef01127aaab968e34f6567e4f2f4
Jul 01 19:47:04.254662 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:04.254622    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4" event={"ID":"3e882572-826f-4b10-9579-e2df0af9cf21","Type":"ContainerStarted","Data":"fc3a543ed2eadcf36b07db7387e7121a368fb6639a61d117732b2a3a8cdf8e8d"}
Jul 01 19:47:04.255168 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:04.254733    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4" event={"ID":"3e882572-826f-4b10-9579-e2df0af9cf21","Type":"ContainerStarted","Data":"c72451e56b70ca940b8a5e971de8b55a5b89ef01127aaab968e34f6567e4f2f4"}
Jul 01 19:47:33.389173 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:33.389130    2581 generic.go:356] "Generic (PLEG): container finished" podID="3e882572-826f-4b10-9579-e2df0af9cf21" containerID="fc3a543ed2eadcf36b07db7387e7121a368fb6639a61d117732b2a3a8cdf8e8d" exitCode=0
Jul 01 19:47:33.389724 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:33.389210    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4" event={"ID":"3e882572-826f-4b10-9579-e2df0af9cf21","Type":"ContainerDied","Data":"fc3a543ed2eadcf36b07db7387e7121a368fb6639a61d117732b2a3a8cdf8e8d"}
Jul 01 19:47:34.395896 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:34.395856    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4" event={"ID":"3e882572-826f-4b10-9579-e2df0af9cf21","Type":"ContainerStarted","Data":"f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773"}
Jul 01 19:47:34.396307 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:34.395980    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4" event={"ID":"3e882572-826f-4b10-9579-e2df0af9cf21","Type":"ContainerStarted","Data":"79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15"}
Jul 01 19:47:34.396307 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:34.396209    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:34.433430 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:34.433375    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4" podStartSLOduration=2.841275489 podStartE2EDuration="32.433361455s" totalImagesPullingTime="0s" totalInitContainerRuntime="29.592085966s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:47:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:47:34.433047135 +0000 UTC m=+1565.668172448" watchObservedRunningTime="2026-07-01 19:47:34.433361455 +0000 UTC m=+1565.668486767"
Jul 01 19:47:35.401861 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:35.401834    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:47:41.413113 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:47:41.413083    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:49:06.213842 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:06.213805    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:49:13.173465 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:13.173426    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"]
Jul 01 19:49:13.173967 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:13.173836    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4" podUID="3e882572-826f-4b10-9579-e2df0af9cf21" containerName="kserve-container" containerID="cri-o://79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15" gracePeriod=30
Jul 01 19:49:13.173967 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:13.173846    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4" podUID="3e882572-826f-4b10-9579-e2df0af9cf21" containerName="kube-rbac-proxy" containerID="cri-o://f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773" gracePeriod=30
Jul 01 19:49:13.870855 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:13.870816    2581 generic.go:356] "Generic (PLEG): container finished" podID="3e882572-826f-4b10-9579-e2df0af9cf21" containerID="f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773" exitCode=2
Jul 01 19:49:13.871062 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:13.870891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4" event={"ID":"3e882572-826f-4b10-9579-e2df0af9cf21","Type":"ContainerDied","Data":"f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773"}
Jul 01 19:49:14.129210 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.129141    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:49:14.217924 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.217889    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59jsw\" (UniqueName: \"kubernetes.io/projected/3e882572-826f-4b10-9579-e2df0af9cf21-kube-api-access-59jsw\") pod \"3e882572-826f-4b10-9579-e2df0af9cf21\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") "
Jul 01 19:49:14.218324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.217958    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3e882572-826f-4b10-9579-e2df0af9cf21-cabundle-cert\") pod \"3e882572-826f-4b10-9579-e2df0af9cf21\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") "
Jul 01 19:49:14.218324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.218070    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3e882572-826f-4b10-9579-e2df0af9cf21-proxy-tls\") pod \"3e882572-826f-4b10-9579-e2df0af9cf21\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") "
Jul 01 19:49:14.218324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.218124    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3e882572-826f-4b10-9579-e2df0af9cf21-kube-rbac-proxy-sar-config\") pod \"3e882572-826f-4b10-9579-e2df0af9cf21\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") "
Jul 01 19:49:14.218324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.218169    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3e882572-826f-4b10-9579-e2df0af9cf21-kserve-provision-location\") pod \"3e882572-826f-4b10-9579-e2df0af9cf21\" (UID: \"3e882572-826f-4b10-9579-e2df0af9cf21\") "
Jul 01 19:49:14.218324 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.218314    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e882572-826f-4b10-9579-e2df0af9cf21-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "3e882572-826f-4b10-9579-e2df0af9cf21" (UID: "3e882572-826f-4b10-9579-e2df0af9cf21"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:49:14.218673 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.218444    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3e882572-826f-4b10-9579-e2df0af9cf21-cabundle-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:49:14.218673 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.218533    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e882572-826f-4b10-9579-e2df0af9cf21-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3e882572-826f-4b10-9579-e2df0af9cf21" (UID: "3e882572-826f-4b10-9579-e2df0af9cf21"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:49:14.220134 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.220107    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e882572-826f-4b10-9579-e2df0af9cf21-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3e882572-826f-4b10-9579-e2df0af9cf21" (UID: "3e882572-826f-4b10-9579-e2df0af9cf21"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:49:14.220134 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.220137    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e882572-826f-4b10-9579-e2df0af9cf21-kube-api-access-59jsw" (OuterVolumeSpecName: "kube-api-access-59jsw") pod "3e882572-826f-4b10-9579-e2df0af9cf21" (UID: "3e882572-826f-4b10-9579-e2df0af9cf21"). InnerVolumeSpecName "kube-api-access-59jsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:49:14.223193 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.223165    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e882572-826f-4b10-9579-e2df0af9cf21-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "3e882572-826f-4b10-9579-e2df0af9cf21" (UID: "3e882572-826f-4b10-9579-e2df0af9cf21"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:49:14.319915 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.319871    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3e882572-826f-4b10-9579-e2df0af9cf21-kserve-provision-location\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:49:14.319915 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.319907    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59jsw\" (UniqueName: \"kubernetes.io/projected/3e882572-826f-4b10-9579-e2df0af9cf21-kube-api-access-59jsw\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:49:14.319915 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.319921    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3e882572-826f-4b10-9579-e2df0af9cf21-proxy-tls\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:49:14.320163 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.319934    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3e882572-826f-4b10-9579-e2df0af9cf21-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:49:14.877178 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.877143    2581 generic.go:356] "Generic (PLEG): container finished" podID="3e882572-826f-4b10-9579-e2df0af9cf21" containerID="79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15" exitCode=0
Jul 01 19:49:14.877356 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.877202    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"
Jul 01 19:49:14.877356 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.877233    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4" event={"ID":"3e882572-826f-4b10-9579-e2df0af9cf21","Type":"ContainerDied","Data":"79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15"}
Jul 01 19:49:14.877356 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.877307    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4" event={"ID":"3e882572-826f-4b10-9579-e2df0af9cf21","Type":"ContainerDied","Data":"c72451e56b70ca940b8a5e971de8b55a5b89ef01127aaab968e34f6567e4f2f4"}
Jul 01 19:49:14.877356 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.877346    2581 scope.go:117] "RemoveContainer" containerID="f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773"
Jul 01 19:49:14.887023 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.887005    2581 scope.go:117] "RemoveContainer" containerID="79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15"
Jul 01 19:49:14.894507 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.894490    2581 scope.go:117] "RemoveContainer" containerID="fc3a543ed2eadcf36b07db7387e7121a368fb6639a61d117732b2a3a8cdf8e8d"
Jul 01 19:49:14.901546 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.901502    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"]
Jul 01 19:49:14.902510 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.902495    2581 scope.go:117] "RemoveContainer" containerID="f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773"
Jul 01 19:49:14.902918 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:49:14.902893    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773\": container with ID starting with f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773 not found: ID does not exist" containerID="f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773"
Jul 01 19:49:14.903047 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.902930    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773"} err="failed to get container status \"f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773\": rpc error: code = NotFound desc = could not find container \"f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773\": container with ID starting with f353e031fe538c0b6c6c3f52d8d57b73e54ecb984fc30dcd6778de647cbfb773 not found: ID does not exist"
Jul 01 19:49:14.903047 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.902956    2581 scope.go:117] "RemoveContainer" containerID="79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15"
Jul 01 19:49:14.903334 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:49:14.903299    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15\": container with ID starting with 79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15 not found: ID does not exist" containerID="79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15"
Jul 01 19:49:14.903420 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.903342    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15"} err="failed to get container status \"79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15\": rpc error: code = NotFound desc = could not find container \"79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15\": container with ID starting with 79fb15fd79a319bae0f9e13ce7e975806c34ce7420eebed110f6f22f22c5db15 not found: ID does not exist"
Jul 01 19:49:14.903420 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.903366    2581 scope.go:117] "RemoveContainer" containerID="fc3a543ed2eadcf36b07db7387e7121a368fb6639a61d117732b2a3a8cdf8e8d"
Jul 01 19:49:14.904084 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:49:14.903894    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc3a543ed2eadcf36b07db7387e7121a368fb6639a61d117732b2a3a8cdf8e8d\": container with ID starting with fc3a543ed2eadcf36b07db7387e7121a368fb6639a61d117732b2a3a8cdf8e8d not found: ID does not exist" containerID="fc3a543ed2eadcf36b07db7387e7121a368fb6639a61d117732b2a3a8cdf8e8d"
Jul 01 19:49:14.904084 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.903925    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc3a543ed2eadcf36b07db7387e7121a368fb6639a61d117732b2a3a8cdf8e8d"} err="failed to get container status \"fc3a543ed2eadcf36b07db7387e7121a368fb6639a61d117732b2a3a8cdf8e8d\": rpc error: code = NotFound desc = could not find container \"fc3a543ed2eadcf36b07db7387e7121a368fb6639a61d117732b2a3a8cdf8e8d\": container with ID starting with fc3a543ed2eadcf36b07db7387e7121a368fb6639a61d117732b2a3a8cdf8e8d not found: ID does not exist"
Jul 01 19:49:14.905695 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:14.905676    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-swgn4"]
Jul 01 19:49:15.215253 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:15.215211    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e882572-826f-4b10-9579-e2df0af9cf21" path="/var/lib/kubelet/pods/3e882572-826f-4b10-9579-e2df0af9cf21/volumes"
Jul 01 19:49:18.312065 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.312024    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"]
Jul 01 19:49:18.312568 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.312545    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e882572-826f-4b10-9579-e2df0af9cf21" containerName="storage-initializer"
Jul 01 19:49:18.312568 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.312588    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e882572-826f-4b10-9579-e2df0af9cf21" containerName="storage-initializer"
Jul 01 19:49:18.312725 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.312608    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e882572-826f-4b10-9579-e2df0af9cf21" containerName="kserve-container"
Jul 01 19:49:18.312725 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.312616    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e882572-826f-4b10-9579-e2df0af9cf21" containerName="kserve-container"
Jul 01 19:49:18.312725 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.312651    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e882572-826f-4b10-9579-e2df0af9cf21" containerName="kube-rbac-proxy"
Jul 01 19:49:18.312725 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.312660    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e882572-826f-4b10-9579-e2df0af9cf21" containerName="kube-rbac-proxy"
Jul 01 19:49:18.312924 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.312763    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e882572-826f-4b10-9579-e2df0af9cf21" containerName="kserve-container"
Jul 01 19:49:18.312924 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.312782    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e882572-826f-4b10-9579-e2df0af9cf21" containerName="kube-rbac-proxy"
Jul 01 19:49:18.317831 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.317808    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.320948 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.320923    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 19:49:18.321050 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.320946    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-kube-rbac-proxy-sar-config\""
Jul 01 19:49:18.321050 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.320952    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 19:49:18.321050 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.320923    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-predictor-serving-cert\""
Jul 01 19:49:18.321483 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.321462    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 19:49:18.321567 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.321513    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 19:49:18.322375 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.322361    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2q28w\""
Jul 01 19:49:18.331443 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.331422    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"]
Jul 01 19:49:18.353947 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.353915    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1327efbe-efe4-4287-ba8d-56c9bfca98c7-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.354118 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.353957    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78r97\" (UniqueName: \"kubernetes.io/projected/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kube-api-access-78r97\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.354118 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.353983    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1327efbe-efe4-4287-ba8d-56c9bfca98c7-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.354118 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.354012    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.354118 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.354035    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.455563 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.455524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.455807 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.455695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1327efbe-efe4-4287-ba8d-56c9bfca98c7-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.455807 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.455727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78r97\" (UniqueName: \"kubernetes.io/projected/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kube-api-access-78r97\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.455807 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.455750    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1327efbe-efe4-4287-ba8d-56c9bfca98c7-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.455807 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.455778    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.456042 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:49:18.455842    2581 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-serving-cert: secret "isvc-autogluon-ts-v1-predictor-serving-cert" not found
Jul 01 19:49:18.456042 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:49:18.455913    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1327efbe-efe4-4287-ba8d-56c9bfca98c7-proxy-tls podName:1327efbe-efe4-4287-ba8d-56c9bfca98c7 nodeName:}" failed. No retries permitted until 2026-07-01 19:49:18.955895452 +0000 UTC m=+1670.191020742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/1327efbe-efe4-4287-ba8d-56c9bfca98c7-proxy-tls") pod "isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" (UID: "1327efbe-efe4-4287-ba8d-56c9bfca98c7") : secret "isvc-autogluon-ts-v1-predictor-serving-cert" not found
Jul 01 19:49:18.456188 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.456152    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.456271 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.456250    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.456371 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.456353    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1327efbe-efe4-4287-ba8d-56c9bfca98c7-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.467297 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.467271    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78r97\" (UniqueName: \"kubernetes.io/projected/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kube-api-access-78r97\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.961124 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.961088    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1327efbe-efe4-4287-ba8d-56c9bfca98c7-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:18.963459 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:18.963434    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1327efbe-efe4-4287-ba8d-56c9bfca98c7-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:19.229392 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:19.229305    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:19.381634 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:19.378947    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"]
Jul 01 19:49:19.387454 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:49:19.387410    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1327efbe_efe4_4287_ba8d_56c9bfca98c7.slice/crio-ce1cc6a107a8f383e81a6b21f03cd79c712d6c50ee44405f50b703f175cb481c WatchSource:0}: Error finding container ce1cc6a107a8f383e81a6b21f03cd79c712d6c50ee44405f50b703f175cb481c: Status 404 returned error can't find the container with id ce1cc6a107a8f383e81a6b21f03cd79c712d6c50ee44405f50b703f175cb481c
Jul 01 19:49:19.904019 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:19.903983    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" event={"ID":"1327efbe-efe4-4287-ba8d-56c9bfca98c7","Type":"ContainerStarted","Data":"a3e13b36822a0321ea255d67ebd3d3a74532f0ee2ad199b5356d9e40147977b2"}
Jul 01 19:49:19.904196 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:19.904058    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" event={"ID":"1327efbe-efe4-4287-ba8d-56c9bfca98c7","Type":"ContainerStarted","Data":"ce1cc6a107a8f383e81a6b21f03cd79c712d6c50ee44405f50b703f175cb481c"}
Jul 01 19:49:28.947361 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:28.947326    2581 generic.go:356] "Generic (PLEG): container finished" podID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerID="a3e13b36822a0321ea255d67ebd3d3a74532f0ee2ad199b5356d9e40147977b2" exitCode=0
Jul 01 19:49:28.947886 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:28.947398    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" event={"ID":"1327efbe-efe4-4287-ba8d-56c9bfca98c7","Type":"ContainerDied","Data":"a3e13b36822a0321ea255d67ebd3d3a74532f0ee2ad199b5356d9e40147977b2"}
Jul 01 19:49:29.954904 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:29.954817    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" event={"ID":"1327efbe-efe4-4287-ba8d-56c9bfca98c7","Type":"ContainerStarted","Data":"c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8"}
Jul 01 19:49:29.955268 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:29.954909    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" event={"ID":"1327efbe-efe4-4287-ba8d-56c9bfca98c7","Type":"ContainerStarted","Data":"3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9"}
Jul 01 19:49:29.955268 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:29.955106    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:29.980431 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:29.980380    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" podStartSLOduration=3.183071545 podStartE2EDuration="11.980364983s" totalImagesPullingTime="0s" totalInitContainerRuntime="8.797293438s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:49:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:49:29.977248497 +0000 UTC m=+1681.212373826" watchObservedRunningTime="2026-07-01 19:49:29.980364983 +0000 UTC m=+1681.215490296"
Jul 01 19:49:30.959545 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:30.959513    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:30.960874 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:30.960841    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 01 19:49:31.964495 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:31.964454    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 01 19:49:36.969807 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:36.969776    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:36.970422 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:36.970389    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 01 19:49:46.970729 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:46.970700    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:48.380563 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:48.380527    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"]
Jul 01 19:49:48.380997 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:48.380875    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="kserve-container" containerID="cri-o://3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9" gracePeriod=30
Jul 01 19:49:48.380997 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:48.380962    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="kube-rbac-proxy" containerID="cri-o://c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8" gracePeriod=30
Jul 01 19:49:49.048916 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.048884    2581 generic.go:356] "Generic (PLEG): container finished" podID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerID="c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8" exitCode=2
Jul 01 19:49:49.049112 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.048921    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" event={"ID":"1327efbe-efe4-4287-ba8d-56c9bfca98c7","Type":"ContainerDied","Data":"c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8"}
Jul 01 19:49:49.323873 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.323846    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:49.436901 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.436861    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1327efbe-efe4-4287-ba8d-56c9bfca98c7-cabundle-cert\") pod \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") "
Jul 01 19:49:49.437378 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.436959    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kube-rbac-proxy-sar-config\") pod \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") "
Jul 01 19:49:49.437378 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.437011    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1327efbe-efe4-4287-ba8d-56c9bfca98c7-proxy-tls\") pod \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") "
Jul 01 19:49:49.437378 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.437084    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kserve-provision-location\") pod \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") "
Jul 01 19:49:49.437378 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.437112    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78r97\" (UniqueName: \"kubernetes.io/projected/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kube-api-access-78r97\") pod \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\" (UID: \"1327efbe-efe4-4287-ba8d-56c9bfca98c7\") "
Jul 01 19:49:49.437378 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.437321    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1327efbe-efe4-4287-ba8d-56c9bfca98c7-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "1327efbe-efe4-4287-ba8d-56c9bfca98c7" (UID: "1327efbe-efe4-4287-ba8d-56c9bfca98c7"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:49:49.437378 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.437346    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1327efbe-efe4-4287-ba8d-56c9bfca98c7" (UID: "1327efbe-efe4-4287-ba8d-56c9bfca98c7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 19:49:49.438096 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.438071    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1327efbe-efe4-4287-ba8d-56c9bfca98c7" (UID: "1327efbe-efe4-4287-ba8d-56c9bfca98c7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 19:49:49.439201 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.439180    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1327efbe-efe4-4287-ba8d-56c9bfca98c7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1327efbe-efe4-4287-ba8d-56c9bfca98c7" (UID: "1327efbe-efe4-4287-ba8d-56c9bfca98c7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 19:49:49.439250 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.439194    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kube-api-access-78r97" (OuterVolumeSpecName: "kube-api-access-78r97") pod "1327efbe-efe4-4287-ba8d-56c9bfca98c7" (UID: "1327efbe-efe4-4287-ba8d-56c9bfca98c7"). InnerVolumeSpecName "kube-api-access-78r97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 19:49:49.538698 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.538668    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:49:49.538698 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.538694    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1327efbe-efe4-4287-ba8d-56c9bfca98c7-proxy-tls\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:49:49.538698 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.538704    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kserve-provision-location\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:49:49.538950 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.538714    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78r97\" (UniqueName: \"kubernetes.io/projected/1327efbe-efe4-4287-ba8d-56c9bfca98c7-kube-api-access-78r97\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:49:49.538950 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:49.538722    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1327efbe-efe4-4287-ba8d-56c9bfca98c7-cabundle-cert\") on node \"ip-10-0-138-218.ec2.internal\" DevicePath \"\""
Jul 01 19:49:50.055990 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.055955    2581 generic.go:356] "Generic (PLEG): container finished" podID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerID="3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9" exitCode=0
Jul 01 19:49:50.056164 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.056008    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" event={"ID":"1327efbe-efe4-4287-ba8d-56c9bfca98c7","Type":"ContainerDied","Data":"3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9"}
Jul 01 19:49:50.056164 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.056027    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"
Jul 01 19:49:50.056164 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.056057    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq" event={"ID":"1327efbe-efe4-4287-ba8d-56c9bfca98c7","Type":"ContainerDied","Data":"ce1cc6a107a8f383e81a6b21f03cd79c712d6c50ee44405f50b703f175cb481c"}
Jul 01 19:49:50.056164 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.056084    2581 scope.go:117] "RemoveContainer" containerID="c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8"
Jul 01 19:49:50.064960 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.064939    2581 scope.go:117] "RemoveContainer" containerID="3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9"
Jul 01 19:49:50.072680 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.072662    2581 scope.go:117] "RemoveContainer" containerID="a3e13b36822a0321ea255d67ebd3d3a74532f0ee2ad199b5356d9e40147977b2"
Jul 01 19:49:50.080545 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.080090    2581 scope.go:117] "RemoveContainer" containerID="c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8"
Jul 01 19:49:50.080916 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:49:50.080895    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8\": container with ID starting with c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8 not found: ID does not exist" containerID="c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8"
Jul 01 19:49:50.081023 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.080929    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8"} err="failed to get container status \"c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8\": rpc error: code = NotFound desc = could not find container \"c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8\": container with ID starting with c220d047efa468045557e79691cdf963259ea965ab8f199b5eb1b5faf0ab2cc8 not found: ID does not exist"
Jul 01 19:49:50.081023 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.080955    2581 scope.go:117] "RemoveContainer" containerID="3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9"
Jul 01 19:49:50.081277 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:49:50.081255    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9\": container with ID starting with 3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9 not found: ID does not exist" containerID="3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9"
Jul 01 19:49:50.081321 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.081287    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9"} err="failed to get container status \"3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9\": rpc error: code = NotFound desc = could not find container \"3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9\": container with ID starting with 3fa2ea90431d794bd32488d5e863cff6e999bcd74cd975bcc6dedfd6d08d69e9 not found: ID does not exist"
Jul 01 19:49:50.081364 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.081322    2581 scope.go:117] "RemoveContainer" containerID="a3e13b36822a0321ea255d67ebd3d3a74532f0ee2ad199b5356d9e40147977b2"
Jul 01 19:49:50.081605 ip-10-0-138-218 kubenswrapper[2581]: E0701 19:49:50.081561    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3e13b36822a0321ea255d67ebd3d3a74532f0ee2ad199b5356d9e40147977b2\": container with ID starting with a3e13b36822a0321ea255d67ebd3d3a74532f0ee2ad199b5356d9e40147977b2 not found: ID does not exist" containerID="a3e13b36822a0321ea255d67ebd3d3a74532f0ee2ad199b5356d9e40147977b2"
Jul 01 19:49:50.081704 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.081611    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3e13b36822a0321ea255d67ebd3d3a74532f0ee2ad199b5356d9e40147977b2"} err="failed to get container status \"a3e13b36822a0321ea255d67ebd3d3a74532f0ee2ad199b5356d9e40147977b2\": rpc error: code = NotFound desc = could not find container \"a3e13b36822a0321ea255d67ebd3d3a74532f0ee2ad199b5356d9e40147977b2\": container with ID starting with a3e13b36822a0321ea255d67ebd3d3a74532f0ee2ad199b5356d9e40147977b2 not found: ID does not exist"
Jul 01 19:49:50.082490 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.082462    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"]
Jul 01 19:49:50.087101 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:50.087073    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-mpssq"]
Jul 01 19:49:51.215829 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:49:51.215799    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" path="/var/lib/kubelet/pods/1327efbe-efe4-4287-ba8d-56c9bfca98c7/volumes"
Jul 01 19:50:24.418003 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.417971    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qhllw/must-gather-bqbkn"]
Jul 01 19:50:24.418476 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.418330    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="kserve-container"
Jul 01 19:50:24.418476 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.418342    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="kserve-container"
Jul 01 19:50:24.418476 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.418351    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="kube-rbac-proxy"
Jul 01 19:50:24.418476 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.418356    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="kube-rbac-proxy"
Jul 01 19:50:24.418476 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.418366    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="storage-initializer"
Jul 01 19:50:24.418476 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.418372    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="storage-initializer"
Jul 01 19:50:24.418476 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.418440    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="kube-rbac-proxy"
Jul 01 19:50:24.418476 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.418447    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1327efbe-efe4-4287-ba8d-56c9bfca98c7" containerName="kserve-container"
Jul 01 19:50:24.421763 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.421742    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qhllw/must-gather-bqbkn"
Jul 01 19:50:24.430154 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.430132    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qhllw\"/\"kube-root-ca.crt\""
Jul 01 19:50:24.430901 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.430887    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qhllw\"/\"openshift-service-ca.crt\""
Jul 01 19:50:24.449637 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.449561    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qhllw/must-gather-bqbkn"]
Jul 01 19:50:24.552504 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.552462    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwlsn\" (UniqueName: \"kubernetes.io/projected/51b5e6f9-8afe-46d3-ae06-614ac019cdfc-kube-api-access-fwlsn\") pod \"must-gather-bqbkn\" (UID: \"51b5e6f9-8afe-46d3-ae06-614ac019cdfc\") " pod="openshift-must-gather-qhllw/must-gather-bqbkn"
Jul 01 19:50:24.552700 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.552627    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/51b5e6f9-8afe-46d3-ae06-614ac019cdfc-must-gather-output\") pod \"must-gather-bqbkn\" (UID: \"51b5e6f9-8afe-46d3-ae06-614ac019cdfc\") " pod="openshift-must-gather-qhllw/must-gather-bqbkn"
Jul 01 19:50:24.653476 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.653437    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwlsn\" (UniqueName: \"kubernetes.io/projected/51b5e6f9-8afe-46d3-ae06-614ac019cdfc-kube-api-access-fwlsn\") pod \"must-gather-bqbkn\" (UID: \"51b5e6f9-8afe-46d3-ae06-614ac019cdfc\") " pod="openshift-must-gather-qhllw/must-gather-bqbkn"
Jul 01 19:50:24.653675 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.653512    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/51b5e6f9-8afe-46d3-ae06-614ac019cdfc-must-gather-output\") pod \"must-gather-bqbkn\" (UID: \"51b5e6f9-8afe-46d3-ae06-614ac019cdfc\") " pod="openshift-must-gather-qhllw/must-gather-bqbkn"
Jul 01 19:50:24.653843 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.653826    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/51b5e6f9-8afe-46d3-ae06-614ac019cdfc-must-gather-output\") pod \"must-gather-bqbkn\" (UID: \"51b5e6f9-8afe-46d3-ae06-614ac019cdfc\") " pod="openshift-must-gather-qhllw/must-gather-bqbkn"
Jul 01 19:50:24.663439 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.663406    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwlsn\" (UniqueName: \"kubernetes.io/projected/51b5e6f9-8afe-46d3-ae06-614ac019cdfc-kube-api-access-fwlsn\") pod \"must-gather-bqbkn\" (UID: \"51b5e6f9-8afe-46d3-ae06-614ac019cdfc\") " pod="openshift-must-gather-qhllw/must-gather-bqbkn"
Jul 01 19:50:24.739412 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.739379    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qhllw/must-gather-bqbkn"
Jul 01 19:50:24.878229 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.878201    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qhllw/must-gather-bqbkn"]
Jul 01 19:50:24.880692 ip-10-0-138-218 kubenswrapper[2581]: W0701 19:50:24.880667    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51b5e6f9_8afe_46d3_ae06_614ac019cdfc.slice/crio-3452c83a85d4adaf33bfc83c464395a67a85d2e96705a5eea057e3b0433ecd59 WatchSource:0}: Error finding container 3452c83a85d4adaf33bfc83c464395a67a85d2e96705a5eea057e3b0433ecd59: Status 404 returned error can't find the container with id 3452c83a85d4adaf33bfc83c464395a67a85d2e96705a5eea057e3b0433ecd59
Jul 01 19:50:24.882388 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:24.882371    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 19:50:25.218023 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:25.217989    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qhllw/must-gather-bqbkn" event={"ID":"51b5e6f9-8afe-46d3-ae06-614ac019cdfc","Type":"ContainerStarted","Data":"3452c83a85d4adaf33bfc83c464395a67a85d2e96705a5eea057e3b0433ecd59"}
Jul 01 19:50:26.227042 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:26.226999    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qhllw/must-gather-bqbkn" event={"ID":"51b5e6f9-8afe-46d3-ae06-614ac019cdfc","Type":"ContainerStarted","Data":"c3a692f77dde194ac2a4cb42e04aa92f4637410f1750529746515342ae566008"}
Jul 01 19:50:26.227488 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:26.227056    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qhllw/must-gather-bqbkn" event={"ID":"51b5e6f9-8afe-46d3-ae06-614ac019cdfc","Type":"ContainerStarted","Data":"2798a178f5a09b87acc76e33300522d65707ca83765a2f99a55bf9cfcfc9c569"}
Jul 01 19:50:26.245819 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:26.245771    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qhllw/must-gather-bqbkn" podStartSLOduration=1.65212917 podStartE2EDuration="2.245754523s" totalImagesPullingTime="593.625353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:50:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:50:26.244859794 +0000 UTC m=+1737.479985131" watchObservedRunningTime="2026-07-01 19:50:26.245754523 +0000 UTC m=+1737.480879834"
Jul 01 19:50:27.083041 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:27.083002    2581 ???:1] "http2: server: error reading preface from client 10.0.138.218:59908: read tcp 10.0.138.218:10250->10.0.138.218:59908: read: connection reset by peer"
Jul 01 19:50:27.088380 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:27.088351    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n9vnj_85fa991c-3b94-4f96-ab9b-608b29ad2b20/global-pull-secret-syncer/0.log"
Jul 01 19:50:27.292045 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:27.292020    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-z2d87_c22da25a-46e6-4260-886b-98c2a65d9bbf/konnectivity-agent/0.log"
Jul 01 19:50:27.375911 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:27.375814    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-218.ec2.internal_5bf0f4d1f6b89cb065876b678658105c/haproxy/0.log"
Jul 01 19:50:31.269190 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.269143    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55749f0c-bc29-49cb-9346-9687d3179626/alertmanager/0.log"
Jul 01 19:50:31.304925 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.304848    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55749f0c-bc29-49cb-9346-9687d3179626/config-reloader/0.log"
Jul 01 19:50:31.334643 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.334617    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55749f0c-bc29-49cb-9346-9687d3179626/kube-rbac-proxy-web/0.log"
Jul 01 19:50:31.365321 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.364759    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55749f0c-bc29-49cb-9346-9687d3179626/kube-rbac-proxy/0.log"
Jul 01 19:50:31.399682 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.399553    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55749f0c-bc29-49cb-9346-9687d3179626/kube-rbac-proxy-metric/0.log"
Jul 01 19:50:31.427786 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.427683    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55749f0c-bc29-49cb-9346-9687d3179626/prom-label-proxy/0.log"
Jul 01 19:50:31.454663 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.454630    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_55749f0c-bc29-49cb-9346-9687d3179626/init-config-reloader/0.log"
Jul 01 19:50:31.499830 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.499792    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-c4nbt_e10211ae-3648-4b0b-98ad-70bab9739e9b/cluster-monitoring-operator/0.log"
Jul 01 19:50:31.630959 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.630859    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-69d4cb6966-ww6q2_02b7019b-d5d1-4054-b8f6-4cbf27aa6e1d/metrics-server/0.log"
Jul 01 19:50:31.663156 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.663116    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-ngt9m_340a8dff-386b-4854-b97c-46a3f475e59b/monitoring-plugin/0.log"
Jul 01 19:50:31.806117 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.806085    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x74m2_8476b125-8a53-4baa-b109-cb06bf728010/node-exporter/0.log"
Jul 01 19:50:31.834664 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.834629    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x74m2_8476b125-8a53-4baa-b109-cb06bf728010/kube-rbac-proxy/0.log"
Jul 01 19:50:31.868476 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:31.868370    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x74m2_8476b125-8a53-4baa-b109-cb06bf728010/init-textfile/0.log"
Jul 01 19:50:32.417097 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:32.417058    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-vfzq2_d778352c-e062-42df-9a41-d0e8b084113d/prometheus-operator/0.log"
Jul 01 19:50:32.447282 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:32.447243    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-vfzq2_d778352c-e062-42df-9a41-d0e8b084113d/kube-rbac-proxy/0.log"
Jul 01 19:50:32.482222 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:32.482180    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7ddc8784d4-tj44j_4df0155a-3ee6-4047-8a88-7d0da8422382/prometheus-operator-admission-webhook/0.log"
Jul 01 19:50:32.614393 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:32.614361    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fc8754d75-psxk4_4ab58593-4ff0-41a5-a9f5-2c00f19d1c63/thanos-query/0.log"
Jul 01 19:50:32.641791 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:32.641761    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fc8754d75-psxk4_4ab58593-4ff0-41a5-a9f5-2c00f19d1c63/kube-rbac-proxy-web/0.log"
Jul 01 19:50:32.681959 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:32.681859    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fc8754d75-psxk4_4ab58593-4ff0-41a5-a9f5-2c00f19d1c63/kube-rbac-proxy/0.log"
Jul 01 19:50:32.715935 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:32.715908    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fc8754d75-psxk4_4ab58593-4ff0-41a5-a9f5-2c00f19d1c63/prom-label-proxy/0.log"
Jul 01 19:50:32.747688 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:32.747661    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fc8754d75-psxk4_4ab58593-4ff0-41a5-a9f5-2c00f19d1c63/kube-rbac-proxy-rules/0.log"
Jul 01 19:50:32.779725 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:32.779691    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fc8754d75-psxk4_4ab58593-4ff0-41a5-a9f5-2c00f19d1c63/kube-rbac-proxy-metrics/0.log"
Jul 01 19:50:33.996164 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:33.996129    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cdb5968b9-wc78x_759856dd-e062-4e68-bbb5-9e17011f717e/networking-console-plugin/0.log"
Jul 01 19:50:34.867237 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:34.867213    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5b97df4676-2f4lw_8f3f025d-1666-47a8-8b45-94fd15cce2e3/console/0.log"
Jul 01 19:50:35.346032 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.346003    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"]
Jul 01 19:50:35.352059 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.352031    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.354965 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.354938    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qhllw\"/\"default-dockercfg-vv6n8\""
Jul 01 19:50:35.358165 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.358119    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"]
Jul 01 19:50:35.369475 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.369447    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a30ba0b8-cd9f-4502-b762-7e3e776d3241-lib-modules\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.369652 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.369493    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a30ba0b8-cd9f-4502-b762-7e3e776d3241-sys\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.369652 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.369530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a30ba0b8-cd9f-4502-b762-7e3e776d3241-proc\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.369652 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.369552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a30ba0b8-cd9f-4502-b762-7e3e776d3241-podres\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.369652 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.369589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56gdz\" (UniqueName: \"kubernetes.io/projected/a30ba0b8-cd9f-4502-b762-7e3e776d3241-kube-api-access-56gdz\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.471013 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.470976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a30ba0b8-cd9f-4502-b762-7e3e776d3241-lib-modules\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.471198 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.471029    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a30ba0b8-cd9f-4502-b762-7e3e776d3241-sys\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.471198 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.471074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a30ba0b8-cd9f-4502-b762-7e3e776d3241-proc\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.471198 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.471098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a30ba0b8-cd9f-4502-b762-7e3e776d3241-podres\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.471198 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.471127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56gdz\" (UniqueName: \"kubernetes.io/projected/a30ba0b8-cd9f-4502-b762-7e3e776d3241-kube-api-access-56gdz\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.471198 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.471151    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a30ba0b8-cd9f-4502-b762-7e3e776d3241-lib-modules\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.471198 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.471169    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a30ba0b8-cd9f-4502-b762-7e3e776d3241-sys\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.471445 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.471232    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a30ba0b8-cd9f-4502-b762-7e3e776d3241-proc\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.471445 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.471271    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a30ba0b8-cd9f-4502-b762-7e3e776d3241-podres\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.480997 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.480943    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56gdz\" (UniqueName: \"kubernetes.io/projected/a30ba0b8-cd9f-4502-b762-7e3e776d3241-kube-api-access-56gdz\") pod \"perf-node-gather-daemonset-7kln2\" (UID: \"a30ba0b8-cd9f-4502-b762-7e3e776d3241\") " pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.669202 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.669118    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:35.853186 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:35.853153    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"]
Jul 01 19:50:36.290217 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:36.290184    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2" event={"ID":"a30ba0b8-cd9f-4502-b762-7e3e776d3241","Type":"ContainerStarted","Data":"8c774102ef73651794172cef5a80e8df610d634a5513962e6511a9a6c349b017"}
Jul 01 19:50:36.290417 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:36.290243    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2" event={"ID":"a30ba0b8-cd9f-4502-b762-7e3e776d3241","Type":"ContainerStarted","Data":"6e805c02625dbe3794d640b42d47859d00a7e7783164472e604c662d5392afa3"}
Jul 01 19:50:36.290417 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:36.290347    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:36.307449 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:36.307400    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2" podStartSLOduration=1.307383802 podStartE2EDuration="1.307383802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:50:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:50:36.305069134 +0000 UTC m=+1747.540194460" watchObservedRunningTime="2026-07-01 19:50:36.307383802 +0000 UTC m=+1747.542509114"
Jul 01 19:50:36.373935 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:36.373907    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zvhnr_c9d9a753-e701-4ae6-9c8a-41217d9a8f23/dns/0.log"
Jul 01 19:50:36.397341 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:36.397314    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zvhnr_c9d9a753-e701-4ae6-9c8a-41217d9a8f23/kube-rbac-proxy/0.log"
Jul 01 19:50:36.445263 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:36.445231    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hmg26_78a8e8c5-fc6c-49cf-8726-3b0cf31d8464/dns-node-resolver/0.log"
Jul 01 19:50:36.994118 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:36.994081    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rdn6x_b8556e73-373f-49ed-b16d-a1c51b253357/node-ca/0.log"
Jul 01 19:50:37.772998 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:37.772955    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5dbf8f9b4f-qjprw_8077fede-71c7-4815-93e1-05c14abe96fd/router/0.log"
Jul 01 19:50:38.131162 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:38.131092    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fj7pv_3bc22e1b-c242-4bc2-9a66-f3c5d9b7d8b0/serve-healthcheck-canary/0.log"
Jul 01 19:50:38.512853 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:38.512822    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-gmpj2_5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40/insights-operator/0.log"
Jul 01 19:50:38.512853 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:38.512851    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-gmpj2_5b6f4c0c-c227-4df9-b0b4-26cd3ae87b40/insights-operator/1.log"
Jul 01 19:50:38.594307 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:38.594274    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kq5nv_c135ff2d-fb79-4eca-8c85-5e6a1105f3c2/kube-rbac-proxy/0.log"
Jul 01 19:50:38.613533 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:38.613509    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kq5nv_c135ff2d-fb79-4eca-8c85-5e6a1105f3c2/exporter/0.log"
Jul 01 19:50:38.632163 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:38.632138    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kq5nv_c135ff2d-fb79-4eca-8c85-5e6a1105f3c2/extractor/0.log"
Jul 01 19:50:40.759882 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:40.759846    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-bd65f9df8-llt7t_b9844155-0df5-4ae6-8a2d-628ab0769a2e/manager/0.log"
Jul 01 19:50:40.778844 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:40.778812    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-67d95fdc48-stp9g_2f221aef-6c2e-4e56-9903-99cde5bf0f98/manager/0.log"
Jul 01 19:50:40.864595 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:40.864517    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-dkrvk_49771f33-bfba-4947-b06c-b1cd7de176f6/manager/0.log"
Jul 01 19:50:42.306151 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:42.306123    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qhllw/perf-node-gather-daemonset-7kln2"
Jul 01 19:50:45.534532 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:45.534495    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-q8bfb_55638e79-3269-468e-a5c2-c8e1acd52a3d/kube-storage-version-migrator-operator/1.log"
Jul 01 19:50:45.535495 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:45.535467    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-q8bfb_55638e79-3269-468e-a5c2-c8e1acd52a3d/kube-storage-version-migrator-operator/0.log"
Jul 01 19:50:46.755395 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:46.755363    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw6c6_314e23aa-a01f-48a2-9b68-ed39d403485c/kube-multus-additional-cni-plugins/0.log"
Jul 01 19:50:46.777114 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:46.777085    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw6c6_314e23aa-a01f-48a2-9b68-ed39d403485c/egress-router-binary-copy/0.log"
Jul 01 19:50:46.797026 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:46.796942    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw6c6_314e23aa-a01f-48a2-9b68-ed39d403485c/cni-plugins/0.log"
Jul 01 19:50:46.817023 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:46.816993    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw6c6_314e23aa-a01f-48a2-9b68-ed39d403485c/bond-cni-plugin/0.log"
Jul 01 19:50:46.837915 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:46.837889    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw6c6_314e23aa-a01f-48a2-9b68-ed39d403485c/routeoverride-cni/0.log"
Jul 01 19:50:46.862607 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:46.862557    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw6c6_314e23aa-a01f-48a2-9b68-ed39d403485c/whereabouts-cni-bincopy/0.log"
Jul 01 19:50:46.884864 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:46.884835    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw6c6_314e23aa-a01f-48a2-9b68-ed39d403485c/whereabouts-cni/0.log"
Jul 01 19:50:46.947186 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:46.947143    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nftfz_b2f63e26-b149-4c9e-b712-69e1add49e6c/kube-multus/0.log"
Jul 01 19:50:47.107079 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:47.106992    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lw425_161462f4-ad95-4e5a-ab54-148c05190926/network-metrics-daemon/0.log"
Jul 01 19:50:47.184698 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:47.184673    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lw425_161462f4-ad95-4e5a-ab54-148c05190926/kube-rbac-proxy/0.log"
Jul 01 19:50:48.435271 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:48.435240    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-controller/0.log"
Jul 01 19:50:48.451013 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:48.450990    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/1.log"
Jul 01 19:50:48.451169 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:48.450993    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovn-acl-logging/0.log"
Jul 01 19:50:48.473077 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:48.473053    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/kube-rbac-proxy-node/0.log"
Jul 01 19:50:48.494374 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:48.494337    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/kube-rbac-proxy-ovn-metrics/0.log"
Jul 01 19:50:48.512681 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:48.512655    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/northd/0.log"
Jul 01 19:50:48.533474 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:48.533449    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/nbdb/0.log"
Jul 01 19:50:48.554237 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:48.554209    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/sbdb/0.log"
Jul 01 19:50:48.601040 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:48.601011    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mhs48_00551914-29ac-4574-a3f3-b5e78dd6dca4/ovnkube-controller/0.log"
Jul 01 19:50:49.546152 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:49.546120    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-56499946bc-2d6jr_c1047257-8b95-4fcc-8c65-c9101d8250c0/check-endpoints/0.log"
Jul 01 19:50:49.567904 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:49.567878    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-595cz_e93ed59e-2406-4e87-a299-26c95b66a0d2/network-check-target-container/0.log"
Jul 01 19:50:50.515068 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:50.515045    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-mdvx6_9d38693f-5c9b-4e27-8894-7b190acc75db/iptables-alerter/0.log"
Jul 01 19:50:51.110913 ip-10-0-138-218 kubenswrapper[2581]: I0701 19:50:51.110879    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-2t8qd_617fc997-a13b-4f86-96ea-378a55762874/tuned/0.log"