Jul 28 16:01:12.323806 ip-10-0-140-224 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 16:01:14.158671 ip-10-0-140-224 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 16:01:14.158671 ip-10-0-140-224 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 16:01:14.158671 ip-10-0-140-224 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 16:01:14.158671 ip-10-0-140-224 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 16:01:14.158671 ip-10-0-140-224 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 16:01:14.159798 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.159708    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 16:01:14.164710 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164691    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 16:01:14.164710 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164710    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164713    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164717    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164720    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164723    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164725    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164728    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164732    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164735    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164738    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164741    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164745    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164747    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164750    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164753    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164755    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164758    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164761    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164763    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164766    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 16:01:14.164782 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164769    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164772    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164774    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164777    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164779    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164782    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164785    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164788    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164791    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164794    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164797    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164799    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164802    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164805    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164808    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164810    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164813    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164816    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164819    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 16:01:14.165280 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164822    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164824    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164827    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164829    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164832    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164835    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164837    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164841    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164843    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164846    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164848    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164850    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164853    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164855    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164858    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164861    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164864    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164866    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164869    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164872    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 16:01:14.165776 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164875    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164877    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164882    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164886    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164889    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164893    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164897    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164900    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164903    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164906    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164909    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164913    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164916    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164919    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164921    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164924    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164926    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164931    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164937    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 16:01:14.166277 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164941    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164946    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164950    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164953    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164957    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164961    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164965    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164967    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164972    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164975    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164978    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164980    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164983    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164985    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164988    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164990    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164993    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164996    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.164998    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.165001    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 16:01:14.166742 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166457    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166465    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166468    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166472    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166475    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166478    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166481    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166485    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166487    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166490    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166492    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166495    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166497    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166500    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166502    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166505    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166507    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166510    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166512    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166515    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 16:01:14.167243 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166518    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166521    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166524    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166526    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166529    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166532    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166535    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166537    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166540    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166543    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166545    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166548    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166550    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166553    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166555    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166558    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166560    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166563    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166566    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 16:01:14.167751 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166571    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166575    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166578    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166581    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166584    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166588    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166597    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166600    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166602    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166604    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166607    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166610    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166612    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166616    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166619    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166622    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166625    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166628    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166630    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166632    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 16:01:14.168266 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166635    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166637    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166640    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166644    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166647    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166650    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166652    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166655    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166657    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166660    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166663    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166665    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166669    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166671    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166674    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166676    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166679    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166681    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166684    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 16:01:14.168753 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166686    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166689    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166692    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166694    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166697    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166700    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166702    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166705    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166708    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166710    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166713    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166716    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166718    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166721    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166723    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166726    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166729    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166731    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166734    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166743    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 16:01:14.169238 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.166746    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167152    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167158    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167164    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167166    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167170    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167173    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167175    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167178    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167181    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167184    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167186    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167189    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167191    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167194    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167197    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167199    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167202    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167205    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 16:01:14.169732 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167207    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167210    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167213    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167215    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167218    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167221    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167223    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167226    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167228    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167231    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167233    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167235    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167238    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167240    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167243    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167245    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167249    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167252    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167255    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167258    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 16:01:14.170208 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167261    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167263    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167266    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167268    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167271    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167273    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167276    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167278    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167280    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167283    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167286    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167289    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167291    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167294    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167296    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167299    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167301    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167305    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167307    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 16:01:14.170689 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167312    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167315    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167318    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167321    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167324    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167326    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167328    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167331    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167334    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167337    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167339    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167342    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167344    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167347    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167350    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167352    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167355    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167357    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167360    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167362    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 16:01:14.171178 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167365    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167367    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167369    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167372    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167375    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167378    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167383    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167386    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167389    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167392    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167395    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167397    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167400    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167403    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167406    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167408    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167411    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167413    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167416    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 16:01:14.171682 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167418    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167421    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.167423    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167497    2575 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167509    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167516    2575 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167521    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167531    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167535    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167543    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167550    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167553    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167556    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167560    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167564    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167567    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167570    2575 flags.go:64] FLAG: --cgroup-root=""
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167575    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167578    2575 flags.go:64] FLAG: --client-ca-file=""
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167581    2575 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167584    2575 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167589    2575 flags.go:64] FLAG: --cluster-domain=""
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167592    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167595    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 16:01:14.172172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167599    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167602    2575 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167606    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167609    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167612    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167615    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167618    2575 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167621    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167624    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167627    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167631    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167635    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167638    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167641    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167644    2575 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167646    2575 flags.go:64] FLAG: --enable-server="true"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167650    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167654    2575 flags.go:64] FLAG: --event-burst="100"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167657    2575 flags.go:64] FLAG: --event-qps="50"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167660    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167663    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167666    2575 flags.go:64] FLAG: --eviction-hard=""
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167670    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167673    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167676    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 16:01:14.172748 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167680    2575 flags.go:64] FLAG: --eviction-soft=""
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167683    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167686    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167689    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167692    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167695    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167698    2575 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167701    2575 flags.go:64] FLAG: --feature-gates=""
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167705    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167708    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167712    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167715    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167718    2575 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167721    2575 flags.go:64] FLAG: --help="false"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167724    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167727    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167730    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167733    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167737    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167740    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167743    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167746    2575 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167749    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 16:01:14.173367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167752    2575 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167758    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167761    2575 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167764    2575 flags.go:64] FLAG: --kube-reserved=""
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167767    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167770    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167773    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167776    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167779    2575 flags.go:64] FLAG: --lock-file=""
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167783    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167786    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167789    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167795    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167798    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167800    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167803    2575 flags.go:64] FLAG: --logging-format="text"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167806    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167810    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167813    2575 flags.go:64] FLAG: --manifest-url=""
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167816    2575 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167820    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167823    2575 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167827    2575 flags.go:64] FLAG: --max-pods="110"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167830    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167833    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 16:01:14.173980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167836    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167840    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167843    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167846    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167849    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167858    2575 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167860    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167863    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167867    2575 flags.go:64] FLAG: --pod-cidr=""
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167871    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167877    2575 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167879    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167883    2575 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167886    2575 flags.go:64] FLAG: --port="10250"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167888    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167891    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cf44d636db543bb1"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167894    2575 flags.go:64] FLAG: --qos-reserved=""
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167899    2575 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167901    2575 flags.go:64] FLAG: --register-node="true"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167904    2575 flags.go:64] FLAG: --register-with-taints=""
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167908    2575 flags.go:64] FLAG: --registry-burst="10"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167911    2575 flags.go:64] FLAG: --registry-qps="5"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167914    2575 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167916    2575 flags.go:64] FLAG: --reserved-memory=""
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167920    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 16:01:14.174654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167923    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167926    2575 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167929    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167932    2575 flags.go:64] FLAG: --runonce="false"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167935    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167938    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167941    2575 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167944    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167950    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167954    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167957    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167960    2575 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167962    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167965    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167968    2575 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167971    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167974    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167977    2575 flags.go:64] FLAG: --system-cgroups=""
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167980    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167985    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167988    2575 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167990    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167995    2575 flags.go:64] FLAG: --tls-min-version=""
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.167998    2575 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.168002    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 16:01:14.175300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.168005    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.168008    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.168011    2575 flags.go:64] FLAG: --v="2"
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.168016    2575 flags.go:64] FLAG: --version="false"
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.168020    2575 flags.go:64] FLAG: --vmodule=""
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.168024    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.168027    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168156    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168160    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168163    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168166    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168169    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168172    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168174    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168177    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168181    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168184    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168187    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168189    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168192    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168195    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 16:01:14.175917 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168197    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168200    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168202    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168205    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168208    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168211    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168214    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168216    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168219    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168221    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168225    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168229    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168233    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168236    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168239    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168242    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168245    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168247    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168250    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 16:01:14.176637 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168252    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168255    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168257    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168260    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168264    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168268    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168271    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168276    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168278    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168281    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168284    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168286    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168289    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168292    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168294    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168297    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168299    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168302    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168306    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168308    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 16:01:14.177345 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168311    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168313    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168316    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168319    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168322    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168324    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168327    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168329    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168332    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168335    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168337    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168340    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168342    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168345    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168348    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168350    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168353    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168355    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168358    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168362    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 16:01:14.177869 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168365    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168369    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168372    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168374    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168377    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168380    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168383    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168386    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168388    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168391    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168394    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168397    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168400    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168402    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168405    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168408    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168410    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168413    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168415    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168418    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 16:01:14.178369 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168420    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168423    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168425    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168428    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168430    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.168433    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.169112    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.177519    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.177538    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.178384    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.178397    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178463    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178468    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178472    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178476    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 16:01:14.178860 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178478    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178482    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178486    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178490    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178493    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178496    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178499    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178502    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178505    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178507    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178510    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178513    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178516    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178519    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178521    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178524    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178528    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178530    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178533    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 16:01:14.179250 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178536    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178538    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178541    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178544    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178547    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178550    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178553    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178555    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178558    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178561    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178563    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178566    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178568    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178572    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178576    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178579    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178581    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178584    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178587    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178590    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 16:01:14.179696 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178593    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178596    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178598    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178603    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178607    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178610    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178612    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178615    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178618    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178621    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178625    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178627    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178630    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178632    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178636    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178638    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178641    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178644    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178647    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 16:01:14.180210 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178650    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178652    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178655    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178657    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178660    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178663    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178666    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178668    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178671    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178674    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178676    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178679    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178681    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178684    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178686    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178689    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178692    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178696    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178699    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178702    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 16:01:14.180666 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178704    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178707    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178710    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178713    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178715    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178718    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178720    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178723    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178725    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178728    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178732    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178734    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178745    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178748    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178750    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178753    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 16:01:14.181289 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178756    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.178761    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178880    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178886    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178889    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178892    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178896    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178898    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178901    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178904    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178907    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178910    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178912    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178915    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178919    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 16:01:14.181726 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178922    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178925    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178927    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178930    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178933    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178935    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178938    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178940    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178943    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178946    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178948    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178952    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178955    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178958    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178960    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178963    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178968    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178972    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178975    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 16:01:14.182111 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178978    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178981    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178984    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178986    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178989    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178992    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178995    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.178997    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179000    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179003    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179006    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179008    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179011    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179014    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179017    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179019    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179022    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179024    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179027    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179030    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 16:01:14.182587 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179032    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179035    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179037    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179040    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179043    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179045    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179048    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179051    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179053    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179056    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179058    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179061    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179064    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179066    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179069    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179071    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179074    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179076    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179079    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179081    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 16:01:14.183081 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179084    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179087    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179090    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179109    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179113    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179116    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179120    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179123    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179126    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179128    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179131    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179134    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179137    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179140    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179142    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179145    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179149    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179153    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179156    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 16:01:14.183583 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179159    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179161    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179164    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179167    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179169    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179172    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179175    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:14.179177    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.179182    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.179312    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.181477    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.183706    2575 server.go:1019] "Starting client certificate rotation"
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.183813    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 16:01:14.184053 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.183852    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 16:01:14.186550 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.186534    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 16:01:14.188104 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.188077    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 16:01:14.194875 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.194854    2575 log.go:25] "Validated CRI v1 runtime API"
Jul 28 16:01:14.201229 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.201215    2575 log.go:25] "Validated CRI v1 image API"
Jul 28 16:01:14.202789 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.202773    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 16:01:14.205351 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.205330    2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a7d07138-232c-484f-bad5-f8e9c1fa9d63:/dev/nvme0n1p4 cfb277b9-b03c-47df-bb5c-d370ccbb51c0:/dev/nvme0n1p3]
Jul 28 16:01:14.205403 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.205351    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 16:01:14.212194 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.211813    2575 manager.go:216] Machine: {Timestamp:2026-07-28 16:01:14.210122607 +0000 UTC m=+0.354501152 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097867 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b76faceebc236791944c7be2f0606 SystemUUID:ec2b76fa-ceeb-c236-7919-44c7be2f0606 BootID:f35aa2c5-b325-4690-84da-06c90576400e Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:61:68:3a:8c:ab Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:61:68:3a:8c:ab Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:41:2c:39:5c:f7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 16:01:14.212194 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.212179    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 16:01:14.212348 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.212250    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 16:01:14.212348 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.212278    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 16:01:14.215146 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.215120    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 16:01:14.215294 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.215150    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-224.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 16:01:14.215883 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.215873    2575 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 16:01:14.215912 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.215886    2575 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 16:01:14.215912 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.215894    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 16:01:14.217285 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.217274    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 16:01:14.218203 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.218193    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 16:01:14.220006 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.219995    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 16:01:14.220136 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.220126    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 16:01:14.220168 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.220155    2575 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 16:01:14.220168 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.220165    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 16:01:14.220225 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.220177    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 16:01:14.220225 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.220187    2575 kubelet.go:398] "Adding apiserver pod source"
Jul 28 16:01:14.220225 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.220196    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 16:01:14.220605 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.220592    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 16:01:14.220651 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.220612    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 16:01:14.223755 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.223741    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 16:01:14.225925 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.225911    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 16:01:14.225984 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.225941    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 16:01:14.227996 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.227983    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 16:01:14.228034 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228004    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 16:01:14.228034 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228012    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 16:01:14.228034 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228018    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 16:01:14.228034 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228027    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 16:01:14.228034 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228032    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 16:01:14.228193 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228041    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 16:01:14.228193 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228047    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 16:01:14.228193 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228055    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 16:01:14.228193 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228070    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 16:01:14.228193 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228085    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 16:01:14.228193 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228103    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 16:01:14.228944 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228924    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 16:01:14.228944 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.228936    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 16:01:14.231106 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.231072    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-224.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 16:01:14.231204 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.231184    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-224.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 16:01:14.231261 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.231187    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 16:01:14.232232 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.232199    2575 server.go:1268] "Started kubelet"
Jul 28 16:01:14.232404 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.232295    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 16:01:14.232512 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.232426    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 16:01:14.232774 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.232723    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 16:01:14.233148 ip-10-0-140-224 systemd[1]: Started Kubernetes Kubelet.
Jul 28 16:01:14.234301 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.234138    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 16:01:14.235045 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.235030    2575 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 16:01:14.236533 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.236513    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c9cr5"
Jul 28 16:01:14.239544 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.239502    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 16:01:14.239639 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.238385    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-224.ec2.internal.18c67f7c524915d9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-224.ec2.internal,UID:ip-10-0-140-224.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-224.ec2.internal,},FirstTimestamp:2026-07-28 16:01:14.232174041 +0000 UTC m=+0.376552586,LastTimestamp:2026-07-28 16:01:14.232174041 +0000 UTC m=+0.376552586,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-224.ec2.internal,}"
Jul 28 16:01:14.241542 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.241528    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 16:01:14.241607 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.241546    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 16:01:14.241924 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.241693    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-224.ec2.internal\" not found"
Jul 28 16:01:14.242212 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.242194    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 16:01:14.242285 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.242224    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 16:01:14.242285 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.242249    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 16:01:14.242285 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.242280    2575 factory.go:55] Registering systemd factory
Jul 28 16:01:14.242431 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.242295    2575 factory.go:223] Registration of the systemd container factory successfully
Jul 28 16:01:14.242431 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.242309    2575 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 16:01:14.242431 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.242319    2575 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 16:01:14.242623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.242599    2575 factory.go:153] Registering CRI-O factory
Jul 28 16:01:14.242676 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.242627    2575 factory.go:223] Registration of the crio container factory successfully
Jul 28 16:01:14.242724 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.242680    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 16:01:14.242724 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.242704    2575 factory.go:103] Registering Raw factory
Jul 28 16:01:14.242724 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.242718    2575 manager.go:1196] Started watching for new ooms in manager
Jul 28 16:01:14.243279 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.243259    2575 manager.go:349] Starting recovery of all containers
Jul 28 16:01:14.244297 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.244268    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c9cr5"
Jul 28 16:01:14.244723 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.244699    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-224.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 16:01:14.244839 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.244819    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 16:01:14.255069 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.255044    2575 manager.go:354] Recovery completed
Jul 28 16:01:14.256450 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.256418    2575 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 28 16:01:14.259627 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.259614    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:01:14.261634 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.261615    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:01:14.261695 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.261646    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:01:14.261695 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.261656    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:01:14.262226 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.262203    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 16:01:14.262226 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.262225    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 16:01:14.262336 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.262215    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:01:14.262336 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.262244    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 16:01:14.262957 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.262938    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:01:14.263037 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.262973    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:01:14.263037 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.262988    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:01:14.263889 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.263819    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-224.ec2.internal.18c67f7c540a9709  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-224.ec2.internal,UID:ip-10-0-140-224.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-224.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-224.ec2.internal,},FirstTimestamp:2026-07-28 16:01:14.261632777 +0000 UTC m=+0.406011321,LastTimestamp:2026-07-28 16:01:14.261632777 +0000 UTC m=+0.406011321,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-224.ec2.internal,}"
Jul 28 16:01:14.264163 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.264141    2575 policy_none.go:49] "None policy: Start"
Jul 28 16:01:14.264163 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.264161    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 16:01:14.264248 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.264172    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 16:01:14.265299 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.265288    2575 policy_none.go:47] "Start"
Jul 28 16:01:14.301717 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.301695    2575 manager.go:337] "Starting Device Plugin manager"
Jul 28 16:01:14.301828 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.301733    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 16:01:14.301828 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.301743    2575 server.go:87] "Starting device plugin registration server"
Jul 28 16:01:14.302017 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.302005    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 16:01:14.302062 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.302021    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 16:01:14.302136 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.302120    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 16:01:14.302459 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.302204    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 16:01:14.302459 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.302415    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 16:01:14.302844 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.302827    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 16:01:14.302903 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.302871    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-224.ec2.internal\" not found"
Jul 28 16:01:14.402328 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.402297    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:01:14.403967 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.403326    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:01:14.403967 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.403356    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:01:14.403967 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.403367    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:01:14.403967 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.403395    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.411911 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.411870    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.411911 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.411898    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-224.ec2.internal\": node \"ip-10-0-140-224.ec2.internal\" not found"
Jul 28 16:01:14.431295 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.431268    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-224.ec2.internal\" not found"
Jul 28 16:01:14.436932 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.436900    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 16:01:14.438247 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.438221    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 16:01:14.438311 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.438258    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 16:01:14.438311 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.438279    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 16:01:14.438380 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.438315    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 16:01:14.440877 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.440854    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 16:01:14.531841 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.531806    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-224.ec2.internal\" not found"
Jul 28 16:01:14.538979 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.538950    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-224.ec2.internal"]
Jul 28 16:01:14.539070 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.539057    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:01:14.540030 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.540013    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:01:14.540147 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.540043    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:01:14.540147 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.540055    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:01:14.541300 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.541288    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:01:14.541411 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.541397    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.541460 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.541425    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:01:14.541947 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.541932    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:01:14.542010 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.541955    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:01:14.542010 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.541971    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:01:14.542081 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.542028    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:01:14.542081 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.542049    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:01:14.542081 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.542058    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:01:14.543355 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.543338    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.543430 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.543355    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7301dfd284c93177b5a9311d5aa95a82-config\") pod \"kube-apiserver-proxy-ip-10-0-140-224.ec2.internal\" (UID: \"7301dfd284c93177b5a9311d5aa95a82\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.543430 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.543363    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:01:14.543430 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.543400    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9aa9e6dd605d0dd717a74a4438dc0149-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal\" (UID: \"9aa9e6dd605d0dd717a74a4438dc0149\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.543430 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.543422    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9aa9e6dd605d0dd717a74a4438dc0149-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal\" (UID: \"9aa9e6dd605d0dd717a74a4438dc0149\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.544107 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.544077    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:01:14.544197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.544125    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:01:14.544197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.544144    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:01:14.557662 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.557642    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-224.ec2.internal\" not found" node="ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.561127 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.561112    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-224.ec2.internal\" not found" node="ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.632281 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.632247    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-224.ec2.internal\" not found"
Jul 28 16:01:14.643615 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.643591    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7301dfd284c93177b5a9311d5aa95a82-config\") pod \"kube-apiserver-proxy-ip-10-0-140-224.ec2.internal\" (UID: \"7301dfd284c93177b5a9311d5aa95a82\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.643753 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.643621    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9aa9e6dd605d0dd717a74a4438dc0149-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal\" (UID: \"9aa9e6dd605d0dd717a74a4438dc0149\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.643753 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.643643    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9aa9e6dd605d0dd717a74a4438dc0149-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal\" (UID: \"9aa9e6dd605d0dd717a74a4438dc0149\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.643753 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.643689    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7301dfd284c93177b5a9311d5aa95a82-config\") pod \"kube-apiserver-proxy-ip-10-0-140-224.ec2.internal\" (UID: \"7301dfd284c93177b5a9311d5aa95a82\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.643753 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.643737    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9aa9e6dd605d0dd717a74a4438dc0149-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal\" (UID: \"9aa9e6dd605d0dd717a74a4438dc0149\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.643880 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.643756    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9aa9e6dd605d0dd717a74a4438dc0149-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal\" (UID: \"9aa9e6dd605d0dd717a74a4438dc0149\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.732800 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.732769    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-224.ec2.internal\" not found"
Jul 28 16:01:14.833298 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.833267    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-224.ec2.internal\" not found"
Jul 28 16:01:14.860756 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.860734    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.865654 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:14.865626    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:14.933891 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:14.933851    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-224.ec2.internal\" not found"
Jul 28 16:01:15.034480 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.034412    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-224.ec2.internal\" not found"
Jul 28 16:01:15.089839 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.089814    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 16:01:15.142120 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.142062    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:15.159495 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.159465    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 16:01:15.161744 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.161729    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-224.ec2.internal"
Jul 28 16:01:15.174528 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.174508    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 16:01:15.182638 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.182614    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 16:01:15.182807 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.182788    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 16:01:15.182865 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.182796    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 16:01:15.220972 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.220940    2575 apiserver.go:52] "Watching apiserver"
Jul 28 16:01:15.235963 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.235931    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 16:01:15.236868 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.236844    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 16:01:15.238025 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.238005    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-lbjpq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal","openshift-network-diagnostics/network-check-target-qknsz","openshift-network-operator/iptables-alerter-nw2gx","kube-system/kube-apiserver-proxy-ip-10-0-140-224.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g","openshift-cluster-node-tuning-operator/tuned-rhq86","openshift-image-registry/node-ca-lkjfw","openshift-multus/multus-additional-cni-plugins-wnqjm","openshift-multus/multus-ll42f","openshift-multus/network-metrics-daemon-f2jjj","openshift-ovn-kubernetes/ovnkube-node-hz6tj","kube-system/global-pull-secret-syncer-9fplh","kube-system/konnectivity-agent-vrwd6"]
Jul 28 16:01:15.239517 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.239500    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.241027 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.241011    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:15.241172 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.241067    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:15.242073 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.242054    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 16:01:15.242183 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.242167    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:15.242255 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.242238    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:15.242854 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.242831    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 16:01:15.243222 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.243195    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 16:01:15.243299 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.243251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nw2gx"
Jul 28 16:01:15.243700 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.243684    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hw442\""
Jul 28 16:01:15.244270 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.244257    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vrwd6"
Jul 28 16:01:15.245315 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.245301    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.246430 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.246416    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.246865 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.246845    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 16:01:15.246865 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.246852    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-sysctl-conf\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.246900    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpv9t\" (UniqueName: \"kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t\") pod \"network-check-target-qknsz\" (UID: \"bd6f7fa4-5752-4b05-a359-9277b293c7b6\") " pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:15.247044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.246908    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 16:01:15.247044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.246928    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/264c7ce5-2d0b-4108-8024-83716b0be5c0-iptables-alerter-script\") pod \"iptables-alerter-nw2gx\" (UID: \"264c7ce5-2d0b-4108-8024-83716b0be5c0\") " pod="openshift-network-operator/iptables-alerter-nw2gx"
Jul 28 16:01:15.247044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.246951    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-systemd\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.246963    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 16:01:15.247044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.246972    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-var-lib-kubelet\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.246978    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-smpt7\""
Jul 28 16:01:15.247044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247016    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:15.247044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247065    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnb87\" (UniqueName: \"kubernetes.io/projected/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-kube-api-access-lnb87\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247116    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pddj\" (UniqueName: \"kubernetes.io/projected/264c7ce5-2d0b-4108-8024-83716b0be5c0-kube-api-access-2pddj\") pod \"iptables-alerter-nw2gx\" (UID: \"264c7ce5-2d0b-4108-8024-83716b0be5c0\") " pod="openshift-network-operator/iptables-alerter-nw2gx"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247142    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-modprobe-d\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247183    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-sysconfig\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247195    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247215    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247211    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-run\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247255    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fd3ce34a-cd54-436a-9331-253d892e271a-etc-tuned\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247269    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247280    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hmkv\" (UniqueName: \"kubernetes.io/projected/fd3ce34a-cd54-436a-9331-253d892e271a-kube-api-access-9hmkv\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247305    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/264c7ce5-2d0b-4108-8024-83716b0be5c0-host-slash\") pod \"iptables-alerter-nw2gx\" (UID: \"264c7ce5-2d0b-4108-8024-83716b0be5c0\") " pod="openshift-network-operator/iptables-alerter-nw2gx"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247327    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-kubernetes\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247349    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-sysctl-d\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247373    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-sys\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247394    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-lib-modules\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247413    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-host\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247426    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd3ce34a-cd54-436a-9331-253d892e271a-tmp\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.247907 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247725    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:15.247907 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.247815    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:15.247973 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.247910    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 16:01:15.248655 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.248644    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 16:01:15.248968 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.248948    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lkjfw"
Jul 28 16:01:15.249592 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.249577    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ms8sd\""
Jul 28 16:01:15.249710 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.249693    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 16:01:15.249710 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.249705    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 16:01:15.249832 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.249734    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 16:01:15.249832 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.249752    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jgl8b\""
Jul 28 16:01:15.249832 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.249763    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 16:01:15.249938 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.249835    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t9nkh\""
Jul 28 16:01:15.249989 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.249978    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 16:01:15.250169 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.250144    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.251140 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.251126    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 16:01:15.251476 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.251462    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lbjpq"
Jul 28 16:01:15.252338 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.252311    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 15:56:14 +0000 UTC" deadline="2028-03-19 17:50:11.45281725 +0000 UTC"
Jul 28 16:01:15.252405 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.252339    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14401h48m56.200482034s"
Jul 28 16:01:15.252405 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.252343    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2m5v7\""
Jul 28 16:01:15.252915 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.252901    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.253492 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.253467    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 16:01:15.253802 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.253782    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 16:01:15.254542 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.254521    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l9sfg\""
Jul 28 16:01:15.254782 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.254755    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 16:01:15.256596 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.256580    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 16:01:15.256718 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.256704    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 16:01:15.256770 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.256723    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 16:01:15.257332 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.257318    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jsdfc\""
Jul 28 16:01:15.257604 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.257588    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 16:01:15.257604 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.257590    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 16:01:15.257729 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.257615    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 16:01:15.258178 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.258164    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-89p94\""
Jul 28 16:01:15.258246 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.258187    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 16:01:15.258246 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.258234    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 16:01:15.264353 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.264336    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 16:01:15.305916 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.305891    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6bx76"
Jul 28 16:01:15.318591 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.318572    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6bx76"
Jul 28 16:01:15.343675 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.343646    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 16:01:15.348344 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348326    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-registration-dir\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.348415 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348351    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-run-systemd\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.348415 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348369    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-node-log\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.348415 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348386    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.348415 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348406    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-host\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.348562 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a724adc-66b2-44b8-8284-216589688dd2-cnibin\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.348562 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348458    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d700d5d0-afa0-407f-b47a-768e795a3949-kubelet-config\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:15.348562 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348509    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w5g5\" (UniqueName: \"kubernetes.io/projected/e3a33ca3-3c32-484a-88b7-8e8401f965fb-kube-api-access-8w5g5\") pod \"node-resolver-lbjpq\" (UID: \"e3a33ca3-3c32-484a-88b7-8e8401f965fb\") " pod="openshift-dns/node-resolver-lbjpq"
Jul 28 16:01:15.348562 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348517    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-host\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.348562 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348533    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnb87\" (UniqueName: \"kubernetes.io/projected/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-kube-api-access-lnb87\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:15.348562 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348548    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-var-lib-cni-bin\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.348766 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348636    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwrtw\" (UniqueName: \"kubernetes.io/projected/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-kube-api-access-qwrtw\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.348766 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348652    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76126427-9dc7-46c3-8120-b0704e288d20-env-overrides\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.348766 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348667    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76126427-9dc7-46c3-8120-b0704e288d20-ovn-node-metrics-cert\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.348766 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348682    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76126427-9dc7-46c3-8120-b0704e288d20-ovnkube-script-lib\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.348766 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348698    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pddj\" (UniqueName: \"kubernetes.io/projected/264c7ce5-2d0b-4108-8024-83716b0be5c0-kube-api-access-2pddj\") pod \"iptables-alerter-nw2gx\" (UID: \"264c7ce5-2d0b-4108-8024-83716b0be5c0\") " pod="openshift-network-operator/iptables-alerter-nw2gx"
Jul 28 16:01:15.348766 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348726    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a724adc-66b2-44b8-8284-216589688dd2-system-cni-dir\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.348766 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348761    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a724adc-66b2-44b8-8284-216589688dd2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.348958 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348781    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7a724adc-66b2-44b8-8284-216589688dd2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.348958 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348796    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:15.348958 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348811    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktj2g\" (UniqueName: \"kubernetes.io/projected/fdddacec-d224-4f6c-b7be-d6e3aa806858-kube-api-access-ktj2g\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.348958 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348826    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-systemd-units\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.348958 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348858    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-run-openvswitch\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.348958 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348888    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-kubernetes\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.348958 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348924    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-sys\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.348958 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348948    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d45a036-19ca-4d55-99d3-2264550e3891-host\") pod \"node-ca-lkjfw\" (UID: \"4d45a036-19ca-4d55-99d3-2264550e3891\") " pod="openshift-image-registry/node-ca-lkjfw"
Jul 28 16:01:15.348958 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348955    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-kubernetes\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348976    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-kubelet\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.348998    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-var-lib-openvswitch\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349006    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-sys\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349015    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-run-ovn\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349033    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-sysctl-conf\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349054    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d700d5d0-afa0-407f-b47a-768e795a3949-dbus\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349073    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-multus-socket-dir-parent\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349087    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e3a33ca3-3c32-484a-88b7-8e8401f965fb-hosts-file\") pod \"node-resolver-lbjpq\" (UID: \"e3a33ca3-3c32-484a-88b7-8e8401f965fb\") " pod="openshift-dns/node-resolver-lbjpq"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349131    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-run-ovn-kubernetes\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349158    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-cni-netd\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349161    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-sysctl-conf\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349177    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-var-lib-kubelet\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.349210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349206    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-system-cni-dir\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349222    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-var-lib-kubelet\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349255    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-hostroot\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349292    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-modprobe-d\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349319    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fd3ce34a-cd54-436a-9331-253d892e271a-etc-tuned\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349341    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a724adc-66b2-44b8-8284-216589688dd2-os-release\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349362    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-cni-binary-copy\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349384    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-var-lib-kubelet\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349398    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-modprobe-d\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349408    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-multus-conf-dir\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349429    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/264c7ce5-2d0b-4108-8024-83716b0be5c0-host-slash\") pod \"iptables-alerter-nw2gx\" (UID: \"264c7ce5-2d0b-4108-8024-83716b0be5c0\") " pod="openshift-network-operator/iptables-alerter-nw2gx"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349447    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-sysctl-d\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-lib-modules\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349466    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/264c7ce5-2d0b-4108-8024-83716b0be5c0-host-slash\") pod \"iptables-alerter-nw2gx\" (UID: \"264c7ce5-2d0b-4108-8024-83716b0be5c0\") " pod="openshift-network-operator/iptables-alerter-nw2gx"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349478    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd3ce34a-cd54-436a-9331-253d892e271a-tmp\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.349558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349536    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-lib-modules\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349564    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-sysctl-d\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349586    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7a724adc-66b2-44b8-8284-216589688dd2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349604    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-run-netns\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349619    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28-konnectivity-ca\") pod \"konnectivity-agent-vrwd6\" (UID: \"ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28\") " pod="kube-system/konnectivity-agent-vrwd6"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349641    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349656    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-var-lib-cni-multus\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349671    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28-agent-certs\") pod \"konnectivity-agent-vrwd6\" (UID: \"ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28\") " pod="kube-system/konnectivity-agent-vrwd6"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349686    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-socket-dir\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349736    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-slash\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349765    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-run-netns\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349806    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-etc-openvswitch\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349838    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-cni-bin\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349871    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs8fh\" (UniqueName: \"kubernetes.io/projected/76126427-9dc7-46c3-8120-b0704e288d20-kube-api-access-xs8fh\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349904    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-systemd\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349921    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:15.350119 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349936    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-etc-kubernetes\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349950    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-device-dir\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349964    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-etc-selinux\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349980    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a33ca3-3c32-484a-88b7-8e8401f965fb-tmp-dir\") pod \"node-resolver-lbjpq\" (UID: \"e3a33ca3-3c32-484a-88b7-8e8401f965fb\") " pod="openshift-dns/node-resolver-lbjpq"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.349994    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hmkv\" (UniqueName: \"kubernetes.io/projected/fd3ce34a-cd54-436a-9331-253d892e271a-kube-api-access-9hmkv\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350014    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-run-k8s-cni-cncf-io\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.350017    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350021    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-systemd\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350036    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-sys-fs\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.350131    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs podName:b549f8a9-6b3c-42bc-bbc8-11d57f9bab80 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:15.850106151 +0000 UTC m=+1.994484699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs") pod "network-metrics-daemon-f2jjj" (UID: "b549f8a9-6b3c-42bc-bbc8-11d57f9bab80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350165    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-log-socket\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350200    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76126427-9dc7-46c3-8120-b0704e288d20-ovnkube-config\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350226    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fgnw\" (UniqueName: \"kubernetes.io/projected/7a724adc-66b2-44b8-8284-216589688dd2-kube-api-access-9fgnw\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350245    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-multus-cni-dir\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350271    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-multus-daemon-config\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350292    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-run-multus-certs\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.350741 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350311    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpv9t\" (UniqueName: \"kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t\") pod \"network-check-target-qknsz\" (UID: \"bd6f7fa4-5752-4b05-a359-9277b293c7b6\") " pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:15.351210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/264c7ce5-2d0b-4108-8024-83716b0be5c0-iptables-alerter-script\") pod \"iptables-alerter-nw2gx\" (UID: \"264c7ce5-2d0b-4108-8024-83716b0be5c0\") " pod="openshift-network-operator/iptables-alerter-nw2gx"
Jul 28 16:01:15.351210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350423    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a724adc-66b2-44b8-8284-216589688dd2-cni-binary-copy\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.351210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350440    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brz5x\" (UniqueName: \"kubernetes.io/projected/4d45a036-19ca-4d55-99d3-2264550e3891-kube-api-access-brz5x\") pod \"node-ca-lkjfw\" (UID: \"4d45a036-19ca-4d55-99d3-2264550e3891\") " pod="openshift-image-registry/node-ca-lkjfw"
Jul 28 16:01:15.351210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350458    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-cnibin\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.351210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350479    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-sysconfig\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.351210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350496    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-run\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.351210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350532    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4d45a036-19ca-4d55-99d3-2264550e3891-serviceca\") pod \"node-ca-lkjfw\" (UID: \"4d45a036-19ca-4d55-99d3-2264550e3891\") " pod="openshift-image-registry/node-ca-lkjfw"
Jul 28 16:01:15.351210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350537    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-etc-sysconfig\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.351210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350575    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-os-release\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.351210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350625    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fd3ce34a-cd54-436a-9331-253d892e271a-run\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.351210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.350727    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 16:01:15.352238 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.352213    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/264c7ce5-2d0b-4108-8024-83716b0be5c0-iptables-alerter-script\") pod \"iptables-alerter-nw2gx\" (UID: \"264c7ce5-2d0b-4108-8024-83716b0be5c0\") " pod="openshift-network-operator/iptables-alerter-nw2gx"
Jul 28 16:01:15.353740 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.353724    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fd3ce34a-cd54-436a-9331-253d892e271a-etc-tuned\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.353845 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.353783    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd3ce34a-cd54-436a-9331-253d892e271a-tmp\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.364274 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.364256    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:01:15.364274 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.364276    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:01:15.364412 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.364287    2575 projected.go:196] Error preparing data for projected volume kube-api-access-vpv9t for pod openshift-network-diagnostics/network-check-target-qknsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:15.364412 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.364353    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t podName:bd6f7fa4-5752-4b05-a359-9277b293c7b6 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:15.864337061 +0000 UTC m=+2.008715592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vpv9t" (UniqueName: "kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t") pod "network-check-target-qknsz" (UID: "bd6f7fa4-5752-4b05-a359-9277b293c7b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:15.366510 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.366487    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pddj\" (UniqueName: \"kubernetes.io/projected/264c7ce5-2d0b-4108-8024-83716b0be5c0-kube-api-access-2pddj\") pod \"iptables-alerter-nw2gx\" (UID: \"264c7ce5-2d0b-4108-8024-83716b0be5c0\") " pod="openshift-network-operator/iptables-alerter-nw2gx"
Jul 28 16:01:15.366652 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.366545    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnb87\" (UniqueName: \"kubernetes.io/projected/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-kube-api-access-lnb87\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:15.367357 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.367340    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hmkv\" (UniqueName: \"kubernetes.io/projected/fd3ce34a-cd54-436a-9331-253d892e271a-kube-api-access-9hmkv\") pod \"tuned-rhq86\" (UID: \"fd3ce34a-cd54-436a-9331-253d892e271a\") " pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.451350 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451313    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a724adc-66b2-44b8-8284-216589688dd2-cnibin\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.451350 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451354    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d700d5d0-afa0-407f-b47a-768e795a3949-kubelet-config\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:15.451635 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451374    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w5g5\" (UniqueName: \"kubernetes.io/projected/e3a33ca3-3c32-484a-88b7-8e8401f965fb-kube-api-access-8w5g5\") pod \"node-resolver-lbjpq\" (UID: \"e3a33ca3-3c32-484a-88b7-8e8401f965fb\") " pod="openshift-dns/node-resolver-lbjpq"
Jul 28 16:01:15.451635 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451398    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-var-lib-cni-bin\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.451635 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451425    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a724adc-66b2-44b8-8284-216589688dd2-cnibin\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.451635 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451437    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-var-lib-cni-bin\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.451635 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451465    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d700d5d0-afa0-407f-b47a-768e795a3949-kubelet-config\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:15.451635 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451519    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwrtw\" (UniqueName: \"kubernetes.io/projected/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-kube-api-access-qwrtw\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.451635 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451554    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76126427-9dc7-46c3-8120-b0704e288d20-env-overrides\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.451635 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451571    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76126427-9dc7-46c3-8120-b0704e288d20-ovn-node-metrics-cert\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.451635 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76126427-9dc7-46c3-8120-b0704e288d20-ovnkube-script-lib\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.451635 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451613    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a724adc-66b2-44b8-8284-216589688dd2-system-cni-dir\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.451635 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451637    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a724adc-66b2-44b8-8284-216589688dd2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451662    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7a724adc-66b2-44b8-8284-216589688dd2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451752    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451787    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktj2g\" (UniqueName: \"kubernetes.io/projected/fdddacec-d224-4f6c-b7be-d6e3aa806858-kube-api-access-ktj2g\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451812    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-systemd-units\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451837    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-run-openvswitch\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451854    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a724adc-66b2-44b8-8284-216589688dd2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.451876    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d45a036-19ca-4d55-99d3-2264550e3891-host\") pod \"node-ca-lkjfw\" (UID: \"4d45a036-19ca-4d55-99d3-2264550e3891\") " pod="openshift-image-registry/node-ca-lkjfw"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451897    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d45a036-19ca-4d55-99d3-2264550e3891-host\") pod \"node-ca-lkjfw\" (UID: \"4d45a036-19ca-4d55-99d3-2264550e3891\") " pod="openshift-image-registry/node-ca-lkjfw"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451919    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-kubelet\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.451937    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret podName:d700d5d0-afa0-407f-b47a-768e795a3949 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:15.951916411 +0000 UTC m=+2.096294957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret") pod "global-pull-secret-syncer-9fplh" (UID: "d700d5d0-afa0-407f-b47a-768e795a3949") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451958    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-kubelet\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451968    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-var-lib-openvswitch\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.451996    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-run-ovn\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452041    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-var-lib-openvswitch\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452071    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d700d5d0-afa0-407f-b47a-768e795a3949-dbus\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:15.452162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452084    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76126427-9dc7-46c3-8120-b0704e288d20-env-overrides\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452129    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-multus-socket-dir-parent\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452136    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-run-ovn\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452151    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-systemd-units\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452181    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e3a33ca3-3c32-484a-88b7-8e8401f965fb-hosts-file\") pod \"node-resolver-lbjpq\" (UID: \"e3a33ca3-3c32-484a-88b7-8e8401f965fb\") " pod="openshift-dns/node-resolver-lbjpq"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452198    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-run-ovn-kubernetes\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452218    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-cni-netd\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452240    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7a724adc-66b2-44b8-8284-216589688dd2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452250    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a724adc-66b2-44b8-8284-216589688dd2-system-cni-dir\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452258    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-run-ovn-kubernetes\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452240    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76126427-9dc7-46c3-8120-b0704e288d20-ovnkube-script-lib\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452272    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-cni-netd\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452245    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d700d5d0-afa0-407f-b47a-768e795a3949-dbus\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452266    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e3a33ca3-3c32-484a-88b7-8e8401f965fb-hosts-file\") pod \"node-resolver-lbjpq\" (UID: \"e3a33ca3-3c32-484a-88b7-8e8401f965fb\") " pod="openshift-dns/node-resolver-lbjpq"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452280    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-system-cni-dir\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452297    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-run-openvswitch\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452309    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-multus-socket-dir-parent\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452316    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-hostroot\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.452983 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452326    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-system-cni-dir\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452344    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a724adc-66b2-44b8-8284-216589688dd2-os-release\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452353    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-hostroot\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452366    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-cni-binary-copy\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452389    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-var-lib-kubelet\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452411    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-multus-conf-dir\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452437    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-var-lib-kubelet\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452440    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7a724adc-66b2-44b8-8284-216589688dd2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452469    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-run-netns\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452476    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-multus-conf-dir\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452439    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a724adc-66b2-44b8-8284-216589688dd2-os-release\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452491    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28-konnectivity-ca\") pod \"konnectivity-agent-vrwd6\" (UID: \"ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28\") " pod="kube-system/konnectivity-agent-vrwd6"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452522    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-run-netns\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452559    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452585    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-var-lib-cni-multus\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452610    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28-agent-certs\") pod \"konnectivity-agent-vrwd6\" (UID: \"ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28\") " pod="kube-system/konnectivity-agent-vrwd6"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452655    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-var-lib-cni-multus\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452716    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-socket-dir\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.453856 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452771    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-slash\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452904    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-socket-dir\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452945    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-cni-binary-copy\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452960    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7a724adc-66b2-44b8-8284-216589688dd2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.452956    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-run-netns\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453021    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-etc-openvswitch\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453072    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-cni-bin\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453077    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-slash\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453086    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-run-netns\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453119    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs8fh\" (UniqueName: \"kubernetes.io/projected/76126427-9dc7-46c3-8120-b0704e288d20-kube-api-access-xs8fh\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453145    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-cni-bin\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453117    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28-konnectivity-ca\") pod \"konnectivity-agent-vrwd6\" (UID: \"ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28\") " pod="kube-system/konnectivity-agent-vrwd6"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453159    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-etc-kubernetes\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453198    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-device-dir\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453205    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-etc-kubernetes\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453222    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-etc-selinux\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453256    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-device-dir\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.454490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453256    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a33ca3-3c32-484a-88b7-8e8401f965fb-tmp-dir\") pod \"node-resolver-lbjpq\" (UID: \"e3a33ca3-3c32-484a-88b7-8e8401f965fb\") " pod="openshift-dns/node-resolver-lbjpq"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453298    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-run-k8s-cni-cncf-io\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453324    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-sys-fs\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453349    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-log-socket\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453351    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-run-k8s-cni-cncf-io\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453364    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-etc-selinux\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453373    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76126427-9dc7-46c3-8120-b0704e288d20-ovnkube-config\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453399    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-log-socket\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453399    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fgnw\" (UniqueName: \"kubernetes.io/projected/7a724adc-66b2-44b8-8284-216589688dd2-kube-api-access-9fgnw\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-sys-fs\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-multus-cni-dir\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453450    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-multus-daemon-config\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453475    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-run-multus-certs\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453503    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-multus-cni-dir\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453518    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a724adc-66b2-44b8-8284-216589688dd2-cni-binary-copy\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453548    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brz5x\" (UniqueName: \"kubernetes.io/projected/4d45a036-19ca-4d55-99d3-2264550e3891-kube-api-access-brz5x\") pod \"node-ca-lkjfw\" (UID: \"4d45a036-19ca-4d55-99d3-2264550e3891\") " pod="openshift-image-registry/node-ca-lkjfw"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453572    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-cnibin\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453584    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a33ca3-3c32-484a-88b7-8e8401f965fb-tmp-dir\") pod \"node-resolver-lbjpq\" (UID: \"e3a33ca3-3c32-484a-88b7-8e8401f965fb\") " pod="openshift-dns/node-resolver-lbjpq"
Jul 28 16:01:15.455028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453598    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4d45a036-19ca-4d55-99d3-2264550e3891-serviceca\") pod \"node-ca-lkjfw\" (UID: \"4d45a036-19ca-4d55-99d3-2264550e3891\") " pod="openshift-image-registry/node-ca-lkjfw"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453733    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-etc-openvswitch\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453816    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-cnibin\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453817    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-os-release\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453836    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-host-run-multus-certs\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453868    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-registration-dir\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453897    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-run-systemd\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453904    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76126427-9dc7-46c3-8120-b0704e288d20-ovnkube-config\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453911    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-registration-dir\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453870    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-os-release\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453921    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-node-log\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453969    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-run-systemd\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453968    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a724adc-66b2-44b8-8284-216589688dd2-cni-binary-copy\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.454009    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-node-log\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.453968    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.454063    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fdddacec-d224-4f6c-b7be-d6e3aa806858-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.454013    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76126427-9dc7-46c3-8120-b0704e288d20-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.455623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.454381    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-multus-daemon-config\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.456319 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.454428    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76126427-9dc7-46c3-8120-b0704e288d20-ovn-node-metrics-cert\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.456319 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.455032    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28-agent-certs\") pod \"konnectivity-agent-vrwd6\" (UID: \"ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28\") " pod="kube-system/konnectivity-agent-vrwd6"
Jul 28 16:01:15.456319 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.455809    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4d45a036-19ca-4d55-99d3-2264550e3891-serviceca\") pod \"node-ca-lkjfw\" (UID: \"4d45a036-19ca-4d55-99d3-2264550e3891\") " pod="openshift-image-registry/node-ca-lkjfw"
Jul 28 16:01:15.462796 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.462770    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwrtw\" (UniqueName: \"kubernetes.io/projected/f95682ff-9c53-4f22-a5ba-b2fa09f213f3-kube-api-access-qwrtw\") pod \"multus-ll42f\" (UID: \"f95682ff-9c53-4f22-a5ba-b2fa09f213f3\") " pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.465277 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.465249    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brz5x\" (UniqueName: \"kubernetes.io/projected/4d45a036-19ca-4d55-99d3-2264550e3891-kube-api-access-brz5x\") pod \"node-ca-lkjfw\" (UID: \"4d45a036-19ca-4d55-99d3-2264550e3891\") " pod="openshift-image-registry/node-ca-lkjfw"
Jul 28 16:01:15.465404 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.465330    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktj2g\" (UniqueName: \"kubernetes.io/projected/fdddacec-d224-4f6c-b7be-d6e3aa806858-kube-api-access-ktj2g\") pod \"aws-ebs-csi-driver-node-6zs7g\" (UID: \"fdddacec-d224-4f6c-b7be-d6e3aa806858\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.466015 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.465993    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w5g5\" (UniqueName: \"kubernetes.io/projected/e3a33ca3-3c32-484a-88b7-8e8401f965fb-kube-api-access-8w5g5\") pod \"node-resolver-lbjpq\" (UID: \"e3a33ca3-3c32-484a-88b7-8e8401f965fb\") " pod="openshift-dns/node-resolver-lbjpq"
Jul 28 16:01:15.466443 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.466424    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs8fh\" (UniqueName: \"kubernetes.io/projected/76126427-9dc7-46c3-8120-b0704e288d20-kube-api-access-xs8fh\") pod \"ovnkube-node-hz6tj\" (UID: \"76126427-9dc7-46c3-8120-b0704e288d20\") " pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.467624 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.467605    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fgnw\" (UniqueName: \"kubernetes.io/projected/7a724adc-66b2-44b8-8284-216589688dd2-kube-api-access-9fgnw\") pod \"multus-additional-cni-plugins-wnqjm\" (UID: \"7a724adc-66b2-44b8-8284-216589688dd2\") " pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.505126 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.505081    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 16:01:15.535380 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:15.535341    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aa9e6dd605d0dd717a74a4438dc0149.slice/crio-e05522e923d4ef7754b4247a71c3cf163fdce4b68a6b09d84f89dbb35c1009af WatchSource:0}: Error finding container e05522e923d4ef7754b4247a71c3cf163fdce4b68a6b09d84f89dbb35c1009af: Status 404 returned error can't find the container with id e05522e923d4ef7754b4247a71c3cf163fdce4b68a6b09d84f89dbb35c1009af
Jul 28 16:01:15.540670 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.540648    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 16:01:15.584506 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.584407    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rhq86"
Jul 28 16:01:15.593030 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:15.593004    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd3ce34a_cd54_436a_9331_253d892e271a.slice/crio-5e16a9808589a186bc1ab0ee5e3bd2b20e788b35e57568b968a625218e829f83 WatchSource:0}: Error finding container 5e16a9808589a186bc1ab0ee5e3bd2b20e788b35e57568b968a625218e829f83: Status 404 returned error can't find the container with id 5e16a9808589a186bc1ab0ee5e3bd2b20e788b35e57568b968a625218e829f83
Jul 28 16:01:15.600493 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.600469    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nw2gx"
Jul 28 16:01:15.607557 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:15.607528    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod264c7ce5_2d0b_4108_8024_83716b0be5c0.slice/crio-7c150f41bdd892303da10412f70602edc3ce562822057e3afbc1a1a22aa86b72 WatchSource:0}: Error finding container 7c150f41bdd892303da10412f70602edc3ce562822057e3afbc1a1a22aa86b72: Status 404 returned error can't find the container with id 7c150f41bdd892303da10412f70602edc3ce562822057e3afbc1a1a22aa86b72
Jul 28 16:01:15.617482 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.617460    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vrwd6"
Jul 28 16:01:15.625116 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:15.625073    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff7d1e7e_c218_4b9e_8cfe_b63c9bd54a28.slice/crio-7571d7ca40b53d36e6df7426085170df91c75d01dc2edf79ad6772cbdb1cd272 WatchSource:0}: Error finding container 7571d7ca40b53d36e6df7426085170df91c75d01dc2edf79ad6772cbdb1cd272: Status 404 returned error can't find the container with id 7571d7ca40b53d36e6df7426085170df91c75d01dc2edf79ad6772cbdb1cd272
Jul 28 16:01:15.634271 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.634251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g"
Jul 28 16:01:15.640901 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.640878    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wnqjm"
Jul 28 16:01:15.641165 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:15.641144    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdddacec_d224_4f6c_b7be_d6e3aa806858.slice/crio-0848ae4d1f1e8bb08fdbfe4f8219c6478be9b93af420e35ce2124a6bb3fe67aa WatchSource:0}: Error finding container 0848ae4d1f1e8bb08fdbfe4f8219c6478be9b93af420e35ce2124a6bb3fe67aa: Status 404 returned error can't find the container with id 0848ae4d1f1e8bb08fdbfe4f8219c6478be9b93af420e35ce2124a6bb3fe67aa
Jul 28 16:01:15.682705 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.682651    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lkjfw"
Jul 28 16:01:15.690423 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:15.690396    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d45a036_19ca_4d55_99d3_2264550e3891.slice/crio-7835cac89e1b176badd7705626eafac0fb29243d108882ed30fc34a847b8fbac WatchSource:0}: Error finding container 7835cac89e1b176badd7705626eafac0fb29243d108882ed30fc34a847b8fbac: Status 404 returned error can't find the container with id 7835cac89e1b176badd7705626eafac0fb29243d108882ed30fc34a847b8fbac
Jul 28 16:01:15.696655 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.696634    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ll42f"
Jul 28 16:01:15.703531 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:15.703509    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95682ff_9c53_4f22_a5ba_b2fa09f213f3.slice/crio-851be67dec6f6e753d932f1a5f55d4a14e865ef187c4209ba28d2ee5c6e971e3 WatchSource:0}: Error finding container 851be67dec6f6e753d932f1a5f55d4a14e865ef187c4209ba28d2ee5c6e971e3: Status 404 returned error can't find the container with id 851be67dec6f6e753d932f1a5f55d4a14e865ef187c4209ba28d2ee5c6e971e3
Jul 28 16:01:15.722427 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.722397    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lbjpq"
Jul 28 16:01:15.727584 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.727475    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:15.729873 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:15.729836    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3a33ca3_3c32_484a_88b7_8e8401f965fb.slice/crio-a24a9612b8ad6e8f3313ed851fdb77f080180ef9a86c2ea54f47ba56998cce65 WatchSource:0}: Error finding container a24a9612b8ad6e8f3313ed851fdb77f080180ef9a86c2ea54f47ba56998cce65: Status 404 returned error can't find the container with id a24a9612b8ad6e8f3313ed851fdb77f080180ef9a86c2ea54f47ba56998cce65
Jul 28 16:01:15.735398 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:15.735372    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76126427_9dc7_46c3_8120_b0704e288d20.slice/crio-90b207329f122cd1072ed4f5d6e6c5b0946bf946e1e74aabab26f364b7c1219a WatchSource:0}: Error finding container 90b207329f122cd1072ed4f5d6e6c5b0946bf946e1e74aabab26f364b7c1219a: Status 404 returned error can't find the container with id 90b207329f122cd1072ed4f5d6e6c5b0946bf946e1e74aabab26f364b7c1219a
Jul 28 16:01:15.858275 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.858180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:15.858443 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.858302    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:15.858443 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.858366    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs podName:b549f8a9-6b3c-42bc-bbc8-11d57f9bab80 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:16.858347818 +0000 UTC m=+3.002726358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs") pod "network-metrics-daemon-f2jjj" (UID: "b549f8a9-6b3c-42bc-bbc8-11d57f9bab80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:15.958806 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.958774    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpv9t\" (UniqueName: \"kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t\") pod \"network-check-target-qknsz\" (UID: \"bd6f7fa4-5752-4b05-a359-9277b293c7b6\") " pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:15.958963 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:15.958816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:15.958963 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.958937    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:15.958963 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.958946    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:01:15.958963 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.958964    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:01:15.959140 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.958975    2575 projected.go:196] Error preparing data for projected volume kube-api-access-vpv9t for pod openshift-network-diagnostics/network-check-target-qknsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:15.959140 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.959000    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret podName:d700d5d0-afa0-407f-b47a-768e795a3949 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:16.958976448 +0000 UTC m=+3.103354979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret") pod "global-pull-secret-syncer-9fplh" (UID: "d700d5d0-afa0-407f-b47a-768e795a3949") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:15.959140 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:15.959018    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t podName:bd6f7fa4-5752-4b05-a359-9277b293c7b6 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:16.959006147 +0000 UTC m=+3.103384678 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpv9t" (UniqueName: "kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t") pod "network-check-target-qknsz" (UID: "bd6f7fa4-5752-4b05-a359-9277b293c7b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:16.006379 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:01:16.006347    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7301dfd284c93177b5a9311d5aa95a82.slice/crio-0d521ad62ea571a88e3a661a7e525cf500e618da37b4dde4553fdc931864a6fd WatchSource:0}: Error finding container 0d521ad62ea571a88e3a661a7e525cf500e618da37b4dde4553fdc931864a6fd: Status 404 returned error can't find the container with id 0d521ad62ea571a88e3a661a7e525cf500e618da37b4dde4553fdc931864a6fd
Jul 28 16:01:16.320903 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.320823    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 15:56:15 +0000 UTC" deadline="2028-03-27 07:47:32.48713418 +0000 UTC"
Jul 28 16:01:16.320903 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.320888    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14583h46m16.166250013s"
Jul 28 16:01:16.427989 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.427750    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 16:01:16.439249 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.438631    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:16.439249 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:16.438795    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:16.439786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.439640    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:16.439786 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:16.439743    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:16.448899 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.448847    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ll42f" event={"ID":"f95682ff-9c53-4f22-a5ba-b2fa09f213f3","Type":"ContainerStarted","Data":"851be67dec6f6e753d932f1a5f55d4a14e865ef187c4209ba28d2ee5c6e971e3"}
Jul 28 16:01:16.451002 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.450972    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lkjfw" event={"ID":"4d45a036-19ca-4d55-99d3-2264550e3891","Type":"ContainerStarted","Data":"7835cac89e1b176badd7705626eafac0fb29243d108882ed30fc34a847b8fbac"}
Jul 28 16:01:16.455237 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.455210    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g" event={"ID":"fdddacec-d224-4f6c-b7be-d6e3aa806858","Type":"ContainerStarted","Data":"0848ae4d1f1e8bb08fdbfe4f8219c6478be9b93af420e35ce2124a6bb3fe67aa"}
Jul 28 16:01:16.457475 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.457316    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vrwd6" event={"ID":"ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28","Type":"ContainerStarted","Data":"7571d7ca40b53d36e6df7426085170df91c75d01dc2edf79ad6772cbdb1cd272"}
Jul 28 16:01:16.464508 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.464481    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lbjpq" event={"ID":"e3a33ca3-3c32-484a-88b7-8e8401f965fb","Type":"ContainerStarted","Data":"a24a9612b8ad6e8f3313ed851fdb77f080180ef9a86c2ea54f47ba56998cce65"}
Jul 28 16:01:16.466721 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.466609    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnqjm" event={"ID":"7a724adc-66b2-44b8-8284-216589688dd2","Type":"ContainerStarted","Data":"19b841ae2cade4540daf719876f73abbf387d4bfc8ba5d8422f3fa20911f81bf"}
Jul 28 16:01:16.472240 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.472186    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nw2gx" event={"ID":"264c7ce5-2d0b-4108-8024-83716b0be5c0","Type":"ContainerStarted","Data":"7c150f41bdd892303da10412f70602edc3ce562822057e3afbc1a1a22aa86b72"}
Jul 28 16:01:16.474676 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.474625    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rhq86" event={"ID":"fd3ce34a-cd54-436a-9331-253d892e271a","Type":"ContainerStarted","Data":"5e16a9808589a186bc1ab0ee5e3bd2b20e788b35e57568b968a625218e829f83"}
Jul 28 16:01:16.477593 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.477524    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal" event={"ID":"9aa9e6dd605d0dd717a74a4438dc0149","Type":"ContainerStarted","Data":"e05522e923d4ef7754b4247a71c3cf163fdce4b68a6b09d84f89dbb35c1009af"}
Jul 28 16:01:16.480935 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.480882    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-224.ec2.internal" event={"ID":"7301dfd284c93177b5a9311d5aa95a82","Type":"ContainerStarted","Data":"0d521ad62ea571a88e3a661a7e525cf500e618da37b4dde4553fdc931864a6fd"}
Jul 28 16:01:16.482739 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.482714    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj" event={"ID":"76126427-9dc7-46c3-8120-b0704e288d20","Type":"ContainerStarted","Data":"90b207329f122cd1072ed4f5d6e6c5b0946bf946e1e74aabab26f364b7c1219a"}
Jul 28 16:01:16.867369 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.867328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:16.867564 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:16.867485    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:16.867564 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:16.867556    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs podName:b549f8a9-6b3c-42bc-bbc8-11d57f9bab80 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:18.867535537 +0000 UTC m=+5.011914077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs") pod "network-metrics-daemon-f2jjj" (UID: "b549f8a9-6b3c-42bc-bbc8-11d57f9bab80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:16.969260 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.968438    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpv9t\" (UniqueName: \"kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t\") pod \"network-check-target-qknsz\" (UID: \"bd6f7fa4-5752-4b05-a359-9277b293c7b6\") " pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:16.969260 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:16.968502    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:16.969260 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:16.968628    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:16.969260 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:16.968694    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret podName:d700d5d0-afa0-407f-b47a-768e795a3949 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:18.968674068 +0000 UTC m=+5.113052620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret") pod "global-pull-secret-syncer-9fplh" (UID: "d700d5d0-afa0-407f-b47a-768e795a3949") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:16.969260 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:16.969141    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:01:16.969260 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:16.969164    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:01:16.969260 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:16.969178    2575 projected.go:196] Error preparing data for projected volume kube-api-access-vpv9t for pod openshift-network-diagnostics/network-check-target-qknsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:16.969260 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:16.969224    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t podName:bd6f7fa4-5752-4b05-a359-9277b293c7b6 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:18.969209004 +0000 UTC m=+5.113587553 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpv9t" (UniqueName: "kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t") pod "network-check-target-qknsz" (UID: "bd6f7fa4-5752-4b05-a359-9277b293c7b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:17.321384 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:17.321335    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 15:56:15 +0000 UTC" deadline="2028-05-15 07:04:38.145725496 +0000 UTC"
Jul 28 16:01:17.321836 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:17.321382    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15759h3m20.824348492s"
Jul 28 16:01:17.367733 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:17.367702    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 16:01:17.439380 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:17.439346    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:17.439574 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:17.439491    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:18.441865 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:18.440988    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:18.441865 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:18.441162    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:18.441865 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:18.441597    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:18.441865 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:18.441693    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:18.885333 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:18.885276    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:18.885533 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:18.885484    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:18.885603 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:18.885550    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs podName:b549f8a9-6b3c-42bc-bbc8-11d57f9bab80 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:22.885529906 +0000 UTC m=+9.029908451 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs") pod "network-metrics-daemon-f2jjj" (UID: "b549f8a9-6b3c-42bc-bbc8-11d57f9bab80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:18.987479 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:18.986689    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:18.987479 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:18.986789    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpv9t\" (UniqueName: \"kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t\") pod \"network-check-target-qknsz\" (UID: \"bd6f7fa4-5752-4b05-a359-9277b293c7b6\") " pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:18.987479 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:18.986939    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:01:18.987479 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:18.986959    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:01:18.987479 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:18.986973    2575 projected.go:196] Error preparing data for projected volume kube-api-access-vpv9t for pod openshift-network-diagnostics/network-check-target-qknsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:18.987479 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:18.987028    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t podName:bd6f7fa4-5752-4b05-a359-9277b293c7b6 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:22.987011229 +0000 UTC m=+9.131389764 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpv9t" (UniqueName: "kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t") pod "network-check-target-qknsz" (UID: "bd6f7fa4-5752-4b05-a359-9277b293c7b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:18.987479 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:18.987114    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:18.987479 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:18.987148    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret podName:d700d5d0-afa0-407f-b47a-768e795a3949 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:22.987137552 +0000 UTC m=+9.131516086 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret") pod "global-pull-secret-syncer-9fplh" (UID: "d700d5d0-afa0-407f-b47a-768e795a3949") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:19.439156 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:19.438964    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:19.439156 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:19.439129    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:20.438609 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:20.438558    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:20.439109 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:20.438696    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:20.439109 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:20.439090    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:20.439221 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:20.439200    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:21.439453 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:21.439418    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:21.439916 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:21.439538    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:22.439977 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:22.439905    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:22.440395 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:22.440038    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:22.440395 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:22.440193    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:22.440395 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:22.440296    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:22.925022 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:22.924884    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:22.925235 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:22.925031    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:22.925235 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:22.925128    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs podName:b549f8a9-6b3c-42bc-bbc8-11d57f9bab80 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:30.925106624 +0000 UTC m=+17.069485171 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs") pod "network-metrics-daemon-f2jjj" (UID: "b549f8a9-6b3c-42bc-bbc8-11d57f9bab80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:23.025986 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:23.025562    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpv9t\" (UniqueName: \"kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t\") pod \"network-check-target-qknsz\" (UID: \"bd6f7fa4-5752-4b05-a359-9277b293c7b6\") " pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:23.025986 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:23.025655    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:23.025986 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:23.025800    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:23.025986 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:23.025873    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret podName:d700d5d0-afa0-407f-b47a-768e795a3949 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:31.025853457 +0000 UTC m=+17.170232004 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret") pod "global-pull-secret-syncer-9fplh" (UID: "d700d5d0-afa0-407f-b47a-768e795a3949") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:23.025986 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:23.025899    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:01:23.025986 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:23.025929    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:01:23.025986 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:23.025942    2575 projected.go:196] Error preparing data for projected volume kube-api-access-vpv9t for pod openshift-network-diagnostics/network-check-target-qknsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:23.025986 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:23.025991    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t podName:bd6f7fa4-5752-4b05-a359-9277b293c7b6 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:31.025975056 +0000 UTC m=+17.170353600 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpv9t" (UniqueName: "kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t") pod "network-check-target-qknsz" (UID: "bd6f7fa4-5752-4b05-a359-9277b293c7b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:23.439009 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:23.438979    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:23.439235 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:23.439136    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:24.439331 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:24.439295    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:24.439792 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:24.439413    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:24.439792 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:24.439484    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:24.439792 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:24.439571    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:25.438782 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:25.438747    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:25.438960 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:25.438893    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:26.438951 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:26.438920    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:26.439406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:26.438920    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:26.439406 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:26.439044    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:26.439406 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:26.439140    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:27.439016 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:27.438991    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:27.439317 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:27.439127    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:28.439299 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:28.439260    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:28.439754 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:28.439267    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:28.439754 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:28.439386    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:28.439754 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:28.439482    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:29.438634 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:29.438590    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:29.438821 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:29.438730    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:30.439112 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:30.438830    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:30.439112 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:30.438958    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:30.439112 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:30.439008    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:30.439681 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:30.439134    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:30.987667 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:30.987629    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:30.987868 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:30.987765    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:30.987868 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:30.987836    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs podName:b549f8a9-6b3c-42bc-bbc8-11d57f9bab80 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:46.987816976 +0000 UTC m=+33.132195506 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs") pod "network-metrics-daemon-f2jjj" (UID: "b549f8a9-6b3c-42bc-bbc8-11d57f9bab80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:31.088345 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:31.088308    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpv9t\" (UniqueName: \"kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t\") pod \"network-check-target-qknsz\" (UID: \"bd6f7fa4-5752-4b05-a359-9277b293c7b6\") " pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:31.088509 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:31.088371    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:31.088509 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:31.088450    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:01:31.088509 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:31.088479    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:01:31.088509 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:31.088494    2575 projected.go:196] Error preparing data for projected volume kube-api-access-vpv9t for pod openshift-network-diagnostics/network-check-target-qknsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:31.088509 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:31.088495    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:31.088788 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:31.088562    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret podName:d700d5d0-afa0-407f-b47a-768e795a3949 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:47.088543996 +0000 UTC m=+33.232922527 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret") pod "global-pull-secret-syncer-9fplh" (UID: "d700d5d0-afa0-407f-b47a-768e795a3949") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:31.088788 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:31.088581    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t podName:bd6f7fa4-5752-4b05-a359-9277b293c7b6 nodeName:}" failed. No retries permitted until 2026-07-28 16:01:47.08857299 +0000 UTC m=+33.232951521 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpv9t" (UniqueName: "kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t") pod "network-check-target-qknsz" (UID: "bd6f7fa4-5752-4b05-a359-9277b293c7b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:31.438684 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:31.438596    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:31.438833 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:31.438725    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:32.439158 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:32.439118    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:32.439738 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:32.439276    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:32.439738 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:32.439118    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:32.439857 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:32.439746    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:33.438775 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:33.438736    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:33.438979 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:33.438877    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:34.440190 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:34.440149    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:34.440638 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:34.440264    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:34.440638 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:34.440325    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:34.440638 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:34.440446    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:35.439288 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:35.439251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:35.439447 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:35.439362    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:36.443903 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.443434    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:36.444662 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.443490    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:36.444662 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:36.443969    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:36.444662 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:36.444015    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:36.534720 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.534676    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rhq86" event={"ID":"fd3ce34a-cd54-436a-9331-253d892e271a","Type":"ContainerStarted","Data":"35f12fc100607717fc75016d998cc361d0ca44447a5ef6c2e62d0fde3a8cba2d"}
Jul 28 16:01:36.536504 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.536473    2575 generic.go:356] "Generic (PLEG): container finished" podID="9aa9e6dd605d0dd717a74a4438dc0149" containerID="0b2c3009563492f469a4e62eb241a5a1fbb3beabb131b0154edb4433496c461e" exitCode=0
Jul 28 16:01:36.536620 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.536555    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal" event={"ID":"9aa9e6dd605d0dd717a74a4438dc0149","Type":"ContainerDied","Data":"0b2c3009563492f469a4e62eb241a5a1fbb3beabb131b0154edb4433496c461e"}
Jul 28 16:01:36.538628 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.538445    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-224.ec2.internal" event={"ID":"7301dfd284c93177b5a9311d5aa95a82","Type":"ContainerStarted","Data":"35effbb62a2eed2b50cdf314741f75bb334cc16d40d042a47af35fa21f4433cd"}
Jul 28 16:01:36.542165 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.542133    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj" event={"ID":"76126427-9dc7-46c3-8120-b0704e288d20","Type":"ContainerStarted","Data":"f720b8c087b8d99d0cd0b4d2c420263d28d5ed43076a7dcfdb2515792208fd32"}
Jul 28 16:01:36.542313 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.542226    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj" event={"ID":"76126427-9dc7-46c3-8120-b0704e288d20","Type":"ContainerStarted","Data":"9fbf858aaa5dd8d6756a7579b1aa920e1214dea684617fb96a684aad0921ac0a"}
Jul 28 16:01:36.542313 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.542278    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj" event={"ID":"76126427-9dc7-46c3-8120-b0704e288d20","Type":"ContainerStarted","Data":"33e350b9c468354d12e7da199b387940ebc5f6324655b23ea9c5ddf2cbe9cf59"}
Jul 28 16:01:36.542417 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.542325    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj" event={"ID":"76126427-9dc7-46c3-8120-b0704e288d20","Type":"ContainerStarted","Data":"d2be9d0ba96091ac74df5fe0f15037241f938e480e362f077f9621d5510fe621"}
Jul 28 16:01:36.542417 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.542370    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj" event={"ID":"76126427-9dc7-46c3-8120-b0704e288d20","Type":"ContainerStarted","Data":"aabecaac2e73ae484d8f7fb57a19023f949ceb0ab943cba1457382103b3ebd0d"}
Jul 28 16:01:36.542492 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.542419    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj" event={"ID":"76126427-9dc7-46c3-8120-b0704e288d20","Type":"ContainerStarted","Data":"0d77e174cb6d4ef0a1b3dccad8946f1751fe1949998988fa543785ef81f1f472"}
Jul 28 16:01:36.543700 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.543672    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ll42f" event={"ID":"f95682ff-9c53-4f22-a5ba-b2fa09f213f3","Type":"ContainerStarted","Data":"c65c0a4d607b08b99e77d96c10e3574fb64dee7705c46cab0fa02f84bb656451"}
Jul 28 16:01:36.545340 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.545316    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lkjfw" event={"ID":"4d45a036-19ca-4d55-99d3-2264550e3891","Type":"ContainerStarted","Data":"82811c7223e4a622111c93f5c36028af1c7736b2a1ef7e2c70df926ca3a04ba0"}
Jul 28 16:01:36.547210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.547183    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g" event={"ID":"fdddacec-d224-4f6c-b7be-d6e3aa806858","Type":"ContainerStarted","Data":"19bc2d7ac9994dc1e820f364d69aeac26f5bb5a8c6c96852fa34f93ea9428192"}
Jul 28 16:01:36.548643 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.548619    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vrwd6" event={"ID":"ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28","Type":"ContainerStarted","Data":"b68f9415153c1e220cd0960a2cfc22e09208c8732ddb398263f8264f54b8e2d6"}
Jul 28 16:01:36.550264 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.550240    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lbjpq" event={"ID":"e3a33ca3-3c32-484a-88b7-8e8401f965fb","Type":"ContainerStarted","Data":"d8a06f97eb9ee9bb666f7fe65d21e04b78074afc6d42d9f51b1cba2cbe0b2121"}
Jul 28 16:01:36.551810 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.551785    2575 generic.go:356] "Generic (PLEG): container finished" podID="7a724adc-66b2-44b8-8284-216589688dd2" containerID="8f0f8fa3bf37681c2df9ea9d174575de26072f2fc26f3361170fb458becf5146" exitCode=0
Jul 28 16:01:36.551923 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.551808    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnqjm" event={"ID":"7a724adc-66b2-44b8-8284-216589688dd2","Type":"ContainerDied","Data":"8f0f8fa3bf37681c2df9ea9d174575de26072f2fc26f3361170fb458becf5146"}
Jul 28 16:01:36.571041 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.570995    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rhq86" podStartSLOduration=2.615094449 podStartE2EDuration="22.570978768s" totalImagesPullingTime="19.955884319s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:01:36.554372054 +0000 UTC m=+22.698750642" watchObservedRunningTime="2026-07-28 16:01:36.570978768 +0000 UTC m=+22.715357393"
Jul 28 16:01:36.571173 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.571115    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lkjfw" podStartSLOduration=10.764020299 podStartE2EDuration="22.571107559s" totalImagesPullingTime="11.80708726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:01:36.570749053 +0000 UTC m=+22.715127636" watchObservedRunningTime="2026-07-28 16:01:36.571107559 +0000 UTC m=+22.715486108"
Jul 28 16:01:36.619025 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.618813    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-224.ec2.internal" podStartSLOduration=21.618797031 podStartE2EDuration="21.618797031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:01:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:01:36.618305375 +0000 UTC m=+22.762683929" watchObservedRunningTime="2026-07-28 16:01:36.618797031 +0000 UTC m=+22.763175800"
Jul 28 16:01:36.619381 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.619348    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ll42f" podStartSLOduration=2.652871935 podStartE2EDuration="22.619338784s" totalImagesPullingTime="19.966466849s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:01:36.598754286 +0000 UTC m=+22.743132838" watchObservedRunningTime="2026-07-28 16:01:36.619338784 +0000 UTC m=+22.763717336"
Jul 28 16:01:36.705229 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.705072    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vrwd6" podStartSLOduration=2.782056441 podStartE2EDuration="22.705054699s" totalImagesPullingTime="19.922998258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:01:36.704865132 +0000 UTC m=+22.849243685" watchObservedRunningTime="2026-07-28 16:01:36.705054699 +0000 UTC m=+22.849433251"
Jul 28 16:01:36.835996 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:36.835969    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lkjfw_4d45a036-19ca-4d55-99d3-2264550e3891/node-ca/0.log"
Jul 28 16:01:37.023679 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:37.023644    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 16:01:37.318830 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:37.318664    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T16:01:37.023669321Z","UUID":"16df7d92-824c-4eac-bc58-773b48941c4d","Handler":null,"Name":"","Endpoint":""}
Jul 28 16:01:37.320578 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:37.320557    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 16:01:37.320723 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:37.320601    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 16:01:37.438706 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:37.438671    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:37.438877 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:37.438825    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:37.556476 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:37.556437    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g" event={"ID":"fdddacec-d224-4f6c-b7be-d6e3aa806858","Type":"ContainerStarted","Data":"a6fe3d55f26cf125069762a7ae5239fdc50168a13a5ed0f3ca4cc140807c1758"}
Jul 28 16:01:37.558266 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:37.558231    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nw2gx" event={"ID":"264c7ce5-2d0b-4108-8024-83716b0be5c0","Type":"ContainerStarted","Data":"68e7ffc231b778538b49e83999b76dff69cc450b448778942e0911f4f76e9bd0"}
Jul 28 16:01:37.560372 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:37.560324    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal" event={"ID":"9aa9e6dd605d0dd717a74a4438dc0149","Type":"ContainerStarted","Data":"df53c4a3b9ba7a76782e4e9144dee55b84f41f65c50bd7facceb504cdc87f8b4"}
Jul 28 16:01:37.585467 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:37.585374    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lbjpq" podStartSLOduration=3.799321746 podStartE2EDuration="23.585353029s" totalImagesPullingTime="19.786031283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:01:36.729728281 +0000 UTC m=+22.874106852" watchObservedRunningTime="2026-07-28 16:01:37.585353029 +0000 UTC m=+23.729731585"
Jul 28 16:01:37.585631 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:37.585518    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nw2gx" podStartSLOduration=3.7027896 podStartE2EDuration="23.585512561s" totalImagesPullingTime="19.882722961s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:01:37.584876176 +0000 UTC m=+23.729254767" watchObservedRunningTime="2026-07-28 16:01:37.585512561 +0000 UTC m=+23.729891114"
Jul 28 16:01:37.604424 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:37.604363    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-224.ec2.internal" podStartSLOduration=22.604341128 podStartE2EDuration="22.604341128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:01:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:01:37.60394372 +0000 UTC m=+23.748322274" watchObservedRunningTime="2026-07-28 16:01:37.604341128 +0000 UTC m=+23.748719682"
Jul 28 16:01:38.442725 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:38.442636    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:38.442725 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:38.442671    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:38.442897 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:38.442767    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:38.442897 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:38.442854    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:38.567729 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:38.567691    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj" event={"ID":"76126427-9dc7-46c3-8120-b0704e288d20","Type":"ContainerStarted","Data":"3a869e42cbc54a4ab5db2f060de4c9596fbdafa3ac77ef06acd0533e45ab1eb4"}
Jul 28 16:01:39.438872 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:39.438830    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:39.439054 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:39.438957    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:39.572539 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:39.572501    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g" event={"ID":"fdddacec-d224-4f6c-b7be-d6e3aa806858","Type":"ContainerStarted","Data":"754031a67fdc47a4dadd0e88d6de8acdb3d73e976b6f8b900540f0f94c932127"}
Jul 28 16:01:40.442814 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:40.442760    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:40.443000 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:40.442887    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:40.443251 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:40.443225    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:40.443393 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:40.443316    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:41.423197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:41.422926    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vrwd6"
Jul 28 16:01:41.423804 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:41.423585    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vrwd6"
Jul 28 16:01:41.438618 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:41.438579    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:41.438790 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:41.438712    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:41.443035 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:41.442988    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6zs7g" podStartSLOduration=5.053106501 podStartE2EDuration="27.442972006s" totalImagesPullingTime="22.389865505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:01:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:01:39.599739977 +0000 UTC m=+25.744118531" watchObservedRunningTime="2026-07-28 16:01:41.442972006 +0000 UTC m=+27.587350563"
Jul 28 16:01:41.580370 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:41.580334    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj" event={"ID":"76126427-9dc7-46c3-8120-b0704e288d20","Type":"ContainerStarted","Data":"656c270de3af27daf78b747ce47b64082fcd23e11c30f8f61ee2489e6e888997"}
Jul 28 16:01:41.580552 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:41.580531    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:41.582205 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:41.582181    2575 generic.go:356] "Generic (PLEG): container finished" podID="7a724adc-66b2-44b8-8284-216589688dd2" containerID="fa3af535e9113714b9d5ff1f730a93714e277c41569137a3212b3e3a2fe595c0" exitCode=0
Jul 28 16:01:41.582340 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:41.582276    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnqjm" event={"ID":"7a724adc-66b2-44b8-8284-216589688dd2","Type":"ContainerDied","Data":"fa3af535e9113714b9d5ff1f730a93714e277c41569137a3212b3e3a2fe595c0"}
Jul 28 16:01:41.582552 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:41.582537    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vrwd6"
Jul 28 16:01:41.583409 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:41.583072    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vrwd6"
Jul 28 16:01:41.597227 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:41.597200    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:41.630271 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:41.630217    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj" podStartSLOduration=7.80602121 podStartE2EDuration="27.630199954s" totalImagesPullingTime="19.824178744s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:01:41.629150121 +0000 UTC m=+27.773528673" watchObservedRunningTime="2026-07-28 16:01:41.630199954 +0000 UTC m=+27.774578506"
Jul 28 16:01:42.439186 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:42.439160    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:42.439735 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:42.439182    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:42.439735 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:42.439306    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:42.439735 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:42.439441    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:42.586996 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:42.586964    2575 generic.go:356] "Generic (PLEG): container finished" podID="7a724adc-66b2-44b8-8284-216589688dd2" containerID="827dfff15618e410d20d84f89176d1599b04f13b0bfee899a3dd375f0379d38c" exitCode=0
Jul 28 16:01:42.587165 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:42.587052    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnqjm" event={"ID":"7a724adc-66b2-44b8-8284-216589688dd2","Type":"ContainerDied","Data":"827dfff15618e410d20d84f89176d1599b04f13b0bfee899a3dd375f0379d38c"}
Jul 28 16:01:42.587376 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:42.587364    2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 16:01:42.588053 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:42.587738    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:42.603838 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:42.603806    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:43.438678 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:43.438639    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:43.438870 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:43.438755    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:43.589439 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:43.589412    2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 16:01:44.442391 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:44.442317    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:44.442530 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:44.442418    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:44.442530 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:44.442471    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:44.442598 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:44.442556    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:44.593978 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:44.593940    2575 generic.go:356] "Generic (PLEG): container finished" podID="7a724adc-66b2-44b8-8284-216589688dd2" containerID="6ef9577d00866ee0da299e738133d3006c646e3d4419b644eff8f0dbfc71aa5c" exitCode=0
Jul 28 16:01:44.594380 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:44.594026    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnqjm" event={"ID":"7a724adc-66b2-44b8-8284-216589688dd2","Type":"ContainerDied","Data":"6ef9577d00866ee0da299e738133d3006c646e3d4419b644eff8f0dbfc71aa5c"}
Jul 28 16:01:44.594380 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:44.594121    2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 16:01:45.439352 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:45.439311    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:45.439535 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:45.439449    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:45.594173 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:45.594135    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:01:45.596218 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:45.596191    2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 16:01:45.612003 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:45.611953    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj" podUID="76126427-9dc7-46c3-8120-b0704e288d20" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 28 16:01:45.622788 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:45.622745    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj" podUID="76126427-9dc7-46c3-8120-b0704e288d20" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 28 16:01:46.439289 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:46.439258    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:46.439480 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:46.439391    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:46.439480 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:46.439449    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:46.439603 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:46.439558    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:47.011296 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:47.011259    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:47.011676 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:47.011384    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:47.011676 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:47.011449    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs podName:b549f8a9-6b3c-42bc-bbc8-11d57f9bab80 nodeName:}" failed. No retries permitted until 2026-07-28 16:02:19.011429632 +0000 UTC m=+65.155808175 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs") pod "network-metrics-daemon-f2jjj" (UID: "b549f8a9-6b3c-42bc-bbc8-11d57f9bab80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:01:47.111618 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:47.111580    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpv9t\" (UniqueName: \"kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t\") pod \"network-check-target-qknsz\" (UID: \"bd6f7fa4-5752-4b05-a359-9277b293c7b6\") " pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:47.111831 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:47.111639    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:47.111831 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:47.111753    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:47.111831 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:47.111775    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:01:47.111831 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:47.111802    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:01:47.111831 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:47.111818    2575 projected.go:196] Error preparing data for projected volume kube-api-access-vpv9t for pod openshift-network-diagnostics/network-check-target-qknsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:47.111831 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:47.111823    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret podName:d700d5d0-afa0-407f-b47a-768e795a3949 nodeName:}" failed. No retries permitted until 2026-07-28 16:02:19.111805408 +0000 UTC m=+65.256183954 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret") pod "global-pull-secret-syncer-9fplh" (UID: "d700d5d0-afa0-407f-b47a-768e795a3949") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:01:47.112146 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:47.111870    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t podName:bd6f7fa4-5752-4b05-a359-9277b293c7b6 nodeName:}" failed. No retries permitted until 2026-07-28 16:02:19.111856414 +0000 UTC m=+65.256234962 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpv9t" (UniqueName: "kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t") pod "network-check-target-qknsz" (UID: "bd6f7fa4-5752-4b05-a359-9277b293c7b6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:01:47.438591 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:47.438514    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:47.438736 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:47.438648    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:48.438583 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:48.438543    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:48.439055 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:48.438543    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:48.439055 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:48.438701    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:48.439055 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:48.438769    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:49.438858 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:49.438825    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:49.439270 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:49.438933    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:49.608678 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:49.608634    2575 generic.go:356] "Generic (PLEG): container finished" podID="7a724adc-66b2-44b8-8284-216589688dd2" containerID="305e080d2e1f600e2d23f7f9a971c4d1d356af6f8f879501ed780c64a1f84eee" exitCode=0
Jul 28 16:01:49.608678 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:49.608680    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnqjm" event={"ID":"7a724adc-66b2-44b8-8284-216589688dd2","Type":"ContainerDied","Data":"305e080d2e1f600e2d23f7f9a971c4d1d356af6f8f879501ed780c64a1f84eee"}
Jul 28 16:01:50.442055 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:50.442028    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:50.442486 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:50.442028    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:50.442486 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:50.442137    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:50.442486 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:50.442227    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:50.613679 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:50.613644    2575 generic.go:356] "Generic (PLEG): container finished" podID="7a724adc-66b2-44b8-8284-216589688dd2" containerID="8ad53988d86112932031d938a65debaf041e4df96200a3d964146a6d5abfb24b" exitCode=0
Jul 28 16:01:50.613854 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:50.613690    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnqjm" event={"ID":"7a724adc-66b2-44b8-8284-216589688dd2","Type":"ContainerDied","Data":"8ad53988d86112932031d938a65debaf041e4df96200a3d964146a6d5abfb24b"}
Jul 28 16:01:51.438861 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:51.438659    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:51.439008 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:51.438949    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:51.619538 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:51.619496    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnqjm" event={"ID":"7a724adc-66b2-44b8-8284-216589688dd2","Type":"ContainerStarted","Data":"b5ab199b69c0df155af74642f04ca109426430a358d45434e08f9b10869e1434"}
Jul 28 16:01:51.648349 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:51.648303    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wnqjm" podStartSLOduration=6.627845736 podStartE2EDuration="37.64828679s" totalImagesPullingTime="31.020441054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:01:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:01:51.64747532 +0000 UTC m=+37.791853873" watchObservedRunningTime="2026-07-28 16:01:51.64828679 +0000 UTC m=+37.792665357"
Jul 28 16:01:52.441714 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:52.441681    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:52.441888 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:52.441779    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:52.441888 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:52.441832    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:52.441971 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:52.441909    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:53.438460 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:53.438430    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:53.438851 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:53.438527    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:54.441509 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:54.441478    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:54.441890 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:54.441584    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:54.441890 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:54.441632    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:54.441890 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:54.441720    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:55.439349 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:55.439310    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:55.439530 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:55.439432    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:56.442087 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:56.442051    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:56.442535 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:56.442203    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:56.442535 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:56.442251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:56.442535 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:56.442334    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:57.438858 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:57.438821    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:57.439027 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:57.438934    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:01:58.439502 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:58.439461    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:01:58.439952 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:58.439522    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:01:58.439952 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:58.439616    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:01:58.439952 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:58.439720    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:01:59.439037 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:01:59.439001    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:01:59.439236 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:01:59.439132    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:02:00.438832 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:00.438796    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:02:00.439236 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:00.438933    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:02:00.439236 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:00.438974    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:02:00.439236 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:00.439065    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:02:01.439488 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:01.439449    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:02:01.439868 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:01.439562    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:02:02.439315 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:02.439273    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:02:02.439516 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:02.439273    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:02:02.439516 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:02.439393    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:02:02.439516 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:02.439475    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:02:03.147881 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:03.147844    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f2jjj"]
Jul 28 16:02:03.148054 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:03.147982    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:02:03.148184 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:03.148129    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:02:03.153452 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:03.153421    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9fplh"]
Jul 28 16:02:03.153623 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:03.153529    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:02:03.153684 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:03.153617    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:02:03.168132 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:03.168073    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qknsz"]
Jul 28 16:02:03.168314 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:03.168267    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:02:03.168404 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:03.168383    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:02:04.440263 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:04.440226    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:02:04.440678 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:04.440330    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:02:04.440678 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:04.440324    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:02:04.440678 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:04.440447    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:02:04.440678 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:04.440456    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:02:04.440678 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:04.440566    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:02:06.438998 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:06.438801    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:02:06.439470 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:06.438811    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:02:06.439470 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:06.439119    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2jjj" podUID="b549f8a9-6b3c-42bc-bbc8-11d57f9bab80"
Jul 28 16:02:06.439470 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:06.438811    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:02:06.439470 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:06.439147    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qknsz" podUID="bd6f7fa4-5752-4b05-a359-9277b293c7b6"
Jul 28 16:02:06.439470 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:06.439225    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9fplh" podUID="d700d5d0-afa0-407f-b47a-768e795a3949"
Jul 28 16:02:07.646281 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.646249    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-224.ec2.internal" event="NodeReady"
Jul 28 16:02:07.646662 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.646368    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 16:02:07.692305 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.692266    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84b869bf79-7dm68"]
Jul 28 16:02:07.694214 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.694188    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.696341 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.696318    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hz5bs\""
Jul 28 16:02:07.696563 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.696544    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 16:02:07.696615 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.696548    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 16:02:07.696786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.696773    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 16:02:07.706351 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.706327    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 16:02:07.708924 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.708901    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l8xg8"]
Jul 28 16:02:07.710722 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.710706    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.713476 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.713448    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 16:02:07.713628 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.713536    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vc5px\""
Jul 28 16:02:07.713628 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.713574    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 16:02:07.713865 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.713846    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84b869bf79-7dm68"]
Jul 28 16:02:07.714358 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.714336    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-scthp"]
Jul 28 16:02:07.716241 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.716221    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-scthp"
Jul 28 16:02:07.718266 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.718243    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fswjp\""
Jul 28 16:02:07.718385 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.718246    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 16:02:07.718456 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.718369    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 16:02:07.718692 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.718671    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 16:02:07.728146 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.728111    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-scthp"]
Jul 28 16:02:07.728711 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.728693    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l8xg8"]
Jul 28 16:02:07.819231 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.819197    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jdnzr"]
Jul 28 16:02:07.822070 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.822052    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:07.824718 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.824693    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 16:02:07.825011 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.824987    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bxxbb\""
Jul 28 16:02:07.825114 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.824988    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 16:02:07.825114 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.824988    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 16:02:07.825201 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.825129    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 16:02:07.834495 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.834466    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jdnzr"]
Jul 28 16:02:07.857228 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857192    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cef1e5c-6531-4245-9341-c6c83e315dbe-ca-trust-extracted\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.857228 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857229    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/157a700c-ac88-481d-a050-7b2e3c0434e9-tmp-dir\") pod \"dns-default-l8xg8\" (UID: \"157a700c-ac88-481d-a050-7b2e3c0434e9\") " pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.857431 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857256    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x65wf\" (UniqueName: \"kubernetes.io/projected/157a700c-ac88-481d-a050-7b2e3c0434e9-kube-api-access-x65wf\") pod \"dns-default-l8xg8\" (UID: \"157a700c-ac88-481d-a050-7b2e3c0434e9\") " pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.857431 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857282    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsnxc\" (UniqueName: \"kubernetes.io/projected/9cef1e5c-6531-4245-9341-c6c83e315dbe-kube-api-access-lsnxc\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.857431 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857360    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cef1e5c-6531-4245-9341-c6c83e315dbe-trusted-ca\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.857431 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857386    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cef1e5c-6531-4245-9341-c6c83e315dbe-registry-tls\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.857431 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857407    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cef1e5c-6531-4245-9341-c6c83e315dbe-image-registry-private-configuration\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.857431 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857425    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cef1e5c-6531-4245-9341-c6c83e315dbe-installation-pull-secrets\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.857610 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857442    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cef1e5c-6531-4245-9341-c6c83e315dbe-bound-sa-token\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.857610 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857473    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/157a700c-ac88-481d-a050-7b2e3c0434e9-metrics-tls\") pod \"dns-default-l8xg8\" (UID: \"157a700c-ac88-481d-a050-7b2e3c0434e9\") " pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.857610 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857494    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cef1e5c-6531-4245-9341-c6c83e315dbe-registry-certificates\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.857610 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857508    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9kbm\" (UniqueName: \"kubernetes.io/projected/60836054-f2ad-4604-a028-77b976a9cf5b-kube-api-access-c9kbm\") pod \"ingress-canary-scthp\" (UID: \"60836054-f2ad-4604-a028-77b976a9cf5b\") " pod="openshift-ingress-canary/ingress-canary-scthp"
Jul 28 16:02:07.857610 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857525    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/157a700c-ac88-481d-a050-7b2e3c0434e9-config-volume\") pod \"dns-default-l8xg8\" (UID: \"157a700c-ac88-481d-a050-7b2e3c0434e9\") " pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.857610 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.857540    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60836054-f2ad-4604-a028-77b976a9cf5b-cert\") pod \"ingress-canary-scthp\" (UID: \"60836054-f2ad-4604-a028-77b976a9cf5b\") " pod="openshift-ingress-canary/ingress-canary-scthp"
Jul 28 16:02:07.958694 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958587    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/157a700c-ac88-481d-a050-7b2e3c0434e9-metrics-tls\") pod \"dns-default-l8xg8\" (UID: \"157a700c-ac88-481d-a050-7b2e3c0434e9\") " pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.958694 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cef1e5c-6531-4245-9341-c6c83e315dbe-registry-certificates\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.958694 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958666    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9kbm\" (UniqueName: \"kubernetes.io/projected/60836054-f2ad-4604-a028-77b976a9cf5b-kube-api-access-c9kbm\") pod \"ingress-canary-scthp\" (UID: \"60836054-f2ad-4604-a028-77b976a9cf5b\") " pod="openshift-ingress-canary/ingress-canary-scthp"
Jul 28 16:02:07.958694 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958686    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp4sm\" (UniqueName: \"kubernetes.io/projected/992fecb8-694b-4cf6-ae74-53488252c696-kube-api-access-qp4sm\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:07.959014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958704    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/157a700c-ac88-481d-a050-7b2e3c0434e9-config-volume\") pod \"dns-default-l8xg8\" (UID: \"157a700c-ac88-481d-a050-7b2e3c0434e9\") " pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.959014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958720    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60836054-f2ad-4604-a028-77b976a9cf5b-cert\") pod \"ingress-canary-scthp\" (UID: \"60836054-f2ad-4604-a028-77b976a9cf5b\") " pod="openshift-ingress-canary/ingress-canary-scthp"
Jul 28 16:02:07.959014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958747    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/992fecb8-694b-4cf6-ae74-53488252c696-crio-socket\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:07.959014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958796    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cef1e5c-6531-4245-9341-c6c83e315dbe-ca-trust-extracted\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.959014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958820    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/157a700c-ac88-481d-a050-7b2e3c0434e9-tmp-dir\") pod \"dns-default-l8xg8\" (UID: \"157a700c-ac88-481d-a050-7b2e3c0434e9\") " pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.959014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x65wf\" (UniqueName: \"kubernetes.io/projected/157a700c-ac88-481d-a050-7b2e3c0434e9-kube-api-access-x65wf\") pod \"dns-default-l8xg8\" (UID: \"157a700c-ac88-481d-a050-7b2e3c0434e9\") " pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.959014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958872    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/992fecb8-694b-4cf6-ae74-53488252c696-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:07.959014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958898    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsnxc\" (UniqueName: \"kubernetes.io/projected/9cef1e5c-6531-4245-9341-c6c83e315dbe-kube-api-access-lsnxc\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.959014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958958    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cef1e5c-6531-4245-9341-c6c83e315dbe-trusted-ca\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.959014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.958986    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cef1e5c-6531-4245-9341-c6c83e315dbe-registry-tls\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.959014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.959010    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/992fecb8-694b-4cf6-ae74-53488252c696-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:07.959378 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.959043    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cef1e5c-6531-4245-9341-c6c83e315dbe-image-registry-private-configuration\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.959378 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.959069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cef1e5c-6531-4245-9341-c6c83e315dbe-installation-pull-secrets\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.959378 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.959114    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cef1e5c-6531-4245-9341-c6c83e315dbe-bound-sa-token\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.959378 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.959139    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/992fecb8-694b-4cf6-ae74-53488252c696-data-volume\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:07.959378 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.959217    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cef1e5c-6531-4245-9341-c6c83e315dbe-ca-trust-extracted\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.959563 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.959500    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/157a700c-ac88-481d-a050-7b2e3c0434e9-config-volume\") pod \"dns-default-l8xg8\" (UID: \"157a700c-ac88-481d-a050-7b2e3c0434e9\") " pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.959729 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.959704    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cef1e5c-6531-4245-9341-c6c83e315dbe-registry-certificates\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.959897 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.959800    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/157a700c-ac88-481d-a050-7b2e3c0434e9-tmp-dir\") pod \"dns-default-l8xg8\" (UID: \"157a700c-ac88-481d-a050-7b2e3c0434e9\") " pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.960362 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.960162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cef1e5c-6531-4245-9341-c6c83e315dbe-trusted-ca\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.963292 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.963258    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/157a700c-ac88-481d-a050-7b2e3c0434e9-metrics-tls\") pod \"dns-default-l8xg8\" (UID: \"157a700c-ac88-481d-a050-7b2e3c0434e9\") " pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.963292 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.963284    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cef1e5c-6531-4245-9341-c6c83e315dbe-image-registry-private-configuration\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.963450 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.963343    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60836054-f2ad-4604-a028-77b976a9cf5b-cert\") pod \"ingress-canary-scthp\" (UID: \"60836054-f2ad-4604-a028-77b976a9cf5b\") " pod="openshift-ingress-canary/ingress-canary-scthp"
Jul 28 16:02:07.963450 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.963368    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cef1e5c-6531-4245-9341-c6c83e315dbe-installation-pull-secrets\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.963450 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.963424    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cef1e5c-6531-4245-9341-c6c83e315dbe-registry-tls\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.972459 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.972429    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9kbm\" (UniqueName: \"kubernetes.io/projected/60836054-f2ad-4604-a028-77b976a9cf5b-kube-api-access-c9kbm\") pod \"ingress-canary-scthp\" (UID: \"60836054-f2ad-4604-a028-77b976a9cf5b\") " pod="openshift-ingress-canary/ingress-canary-scthp"
Jul 28 16:02:07.972459 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.972456    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x65wf\" (UniqueName: \"kubernetes.io/projected/157a700c-ac88-481d-a050-7b2e3c0434e9-kube-api-access-x65wf\") pod \"dns-default-l8xg8\" (UID: \"157a700c-ac88-481d-a050-7b2e3c0434e9\") " pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:07.972650 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.972540    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsnxc\" (UniqueName: \"kubernetes.io/projected/9cef1e5c-6531-4245-9341-c6c83e315dbe-kube-api-access-lsnxc\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:07.973632 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:07.973608    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cef1e5c-6531-4245-9341-c6c83e315dbe-bound-sa-token\") pod \"image-registry-84b869bf79-7dm68\" (UID: \"9cef1e5c-6531-4245-9341-c6c83e315dbe\") " pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:08.007678 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.007636    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:08.022335 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.022295    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:08.026358 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.026338    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-scthp"
Jul 28 16:02:08.059630 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.059586    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/992fecb8-694b-4cf6-ae74-53488252c696-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:08.059793 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.059651    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/992fecb8-694b-4cf6-ae74-53488252c696-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:08.059793 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.059671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/992fecb8-694b-4cf6-ae74-53488252c696-data-volume\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:08.059793 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.059701    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp4sm\" (UniqueName: \"kubernetes.io/projected/992fecb8-694b-4cf6-ae74-53488252c696-kube-api-access-qp4sm\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:08.059793 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.059720    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/992fecb8-694b-4cf6-ae74-53488252c696-crio-socket\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:08.059925 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.059917    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/992fecb8-694b-4cf6-ae74-53488252c696-crio-socket\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:08.060365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.060327    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/992fecb8-694b-4cf6-ae74-53488252c696-data-volume\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:08.060630 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.060608    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/992fecb8-694b-4cf6-ae74-53488252c696-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:08.062837 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.062806    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/992fecb8-694b-4cf6-ae74-53488252c696-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:08.071573 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.071546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp4sm\" (UniqueName: \"kubernetes.io/projected/992fecb8-694b-4cf6-ae74-53488252c696-kube-api-access-qp4sm\") pod \"insights-runtime-extractor-jdnzr\" (UID: \"992fecb8-694b-4cf6-ae74-53488252c696\") " pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:08.131687 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.131594    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jdnzr"
Jul 28 16:02:08.178514 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.178474    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l8xg8"]
Jul 28 16:02:08.182504 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:08.182473    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod157a700c_ac88_481d_a050_7b2e3c0434e9.slice/crio-b1dbf4f4152906b6b71a814984c3675e1b657a3af5f1cc25b96230c8b03e3cef WatchSource:0}: Error finding container b1dbf4f4152906b6b71a814984c3675e1b657a3af5f1cc25b96230c8b03e3cef: Status 404 returned error can't find the container with id b1dbf4f4152906b6b71a814984c3675e1b657a3af5f1cc25b96230c8b03e3cef
Jul 28 16:02:08.183280 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.183252    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84b869bf79-7dm68"]
Jul 28 16:02:08.187803 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:08.187777    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cef1e5c_6531_4245_9341_c6c83e315dbe.slice/crio-2f895072a8621fa9bb86f4a8eb67b5a81839fab61b87421232154c36686b2b04 WatchSource:0}: Error finding container 2f895072a8621fa9bb86f4a8eb67b5a81839fab61b87421232154c36686b2b04: Status 404 returned error can't find the container with id 2f895072a8621fa9bb86f4a8eb67b5a81839fab61b87421232154c36686b2b04
Jul 28 16:02:08.204518 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.201867    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-scthp"]
Jul 28 16:02:08.207977 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:08.207950    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60836054_f2ad_4604_a028_77b976a9cf5b.slice/crio-07993e7bf8306144f9b8d21ec31b50cddcaa0d6b8fe731fb501011b1bf49fe02 WatchSource:0}: Error finding container 07993e7bf8306144f9b8d21ec31b50cddcaa0d6b8fe731fb501011b1bf49fe02: Status 404 returned error can't find the container with id 07993e7bf8306144f9b8d21ec31b50cddcaa0d6b8fe731fb501011b1bf49fe02
Jul 28 16:02:08.288343 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.288312    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jdnzr"]
Jul 28 16:02:08.291480 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:08.291447    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod992fecb8_694b_4cf6_ae74_53488252c696.slice/crio-303f123f754cf65999b97913bb42c8be34b0cc09a79bf7f970024226d28b1656 WatchSource:0}: Error finding container 303f123f754cf65999b97913bb42c8be34b0cc09a79bf7f970024226d28b1656: Status 404 returned error can't find the container with id 303f123f754cf65999b97913bb42c8be34b0cc09a79bf7f970024226d28b1656
Jul 28 16:02:08.438676 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.438633    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:02:08.438829 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.438634    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:02:08.438883 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.438631    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:02:08.443541 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.442939    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 16:02:08.443541 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.443025    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 16:02:08.443541 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.443219    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-84r49\""
Jul 28 16:02:08.444232 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.444022    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kkdj2\""
Jul 28 16:02:08.444232 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.444043    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 16:02:08.444795 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.444452    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 16:02:08.659227 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.659137    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jdnzr" event={"ID":"992fecb8-694b-4cf6-ae74-53488252c696","Type":"ContainerStarted","Data":"303f123f754cf65999b97913bb42c8be34b0cc09a79bf7f970024226d28b1656"}
Jul 28 16:02:08.661325 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.660925    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84b869bf79-7dm68" event={"ID":"9cef1e5c-6531-4245-9341-c6c83e315dbe","Type":"ContainerStarted","Data":"7cea1bfad0cdf8efb5c0bda842e36fbd067b70c4f813ebcb87b64748dd8568f0"}
Jul 28 16:02:08.661325 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.660977    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84b869bf79-7dm68" event={"ID":"9cef1e5c-6531-4245-9341-c6c83e315dbe","Type":"ContainerStarted","Data":"2f895072a8621fa9bb86f4a8eb67b5a81839fab61b87421232154c36686b2b04"}
Jul 28 16:02:08.661325 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.661127    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:08.662367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.662326    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l8xg8" event={"ID":"157a700c-ac88-481d-a050-7b2e3c0434e9","Type":"ContainerStarted","Data":"b1dbf4f4152906b6b71a814984c3675e1b657a3af5f1cc25b96230c8b03e3cef"}
Jul 28 16:02:08.663700 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.663674    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-scthp" event={"ID":"60836054-f2ad-4604-a028-77b976a9cf5b","Type":"ContainerStarted","Data":"07993e7bf8306144f9b8d21ec31b50cddcaa0d6b8fe731fb501011b1bf49fe02"}
Jul 28 16:02:08.684671 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:08.684609    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84b869bf79-7dm68" podStartSLOduration=6.68458842 podStartE2EDuration="6.68458842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:08.683608945 +0000 UTC m=+54.827987501" watchObservedRunningTime="2026-07-28 16:02:08.68458842 +0000 UTC m=+54.828966969"
Jul 28 16:02:12.654181 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:12.654114    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk"]
Jul 28 16:02:12.673684 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:12.673624    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk"]
Jul 28 16:02:12.673852 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:12.673720    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk"
Jul 28 16:02:12.675912 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:12.675869    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 16:02:12.676051 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:12.675956    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-h787g\""
Jul 28 16:02:12.694720 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:12.694682    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1936dcb1-1a31-4466-a3e7-c65e6dada324-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-2gmnk\" (UID: \"1936dcb1-1a31-4466-a3e7-c65e6dada324\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk"
Jul 28 16:02:12.795186 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:12.795142    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1936dcb1-1a31-4466-a3e7-c65e6dada324-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-2gmnk\" (UID: \"1936dcb1-1a31-4466-a3e7-c65e6dada324\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk"
Jul 28 16:02:12.795371 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:12.795305    2575 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 28 16:02:12.795429 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:12.795383    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1936dcb1-1a31-4466-a3e7-c65e6dada324-tls-certificates podName:1936dcb1-1a31-4466-a3e7-c65e6dada324 nodeName:}" failed. No retries permitted until 2026-07-28 16:02:13.295366216 +0000 UTC m=+59.439744747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/1936dcb1-1a31-4466-a3e7-c65e6dada324-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-2gmnk" (UID: "1936dcb1-1a31-4466-a3e7-c65e6dada324") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 28 16:02:13.299109 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:13.299054    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1936dcb1-1a31-4466-a3e7-c65e6dada324-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-2gmnk\" (UID: \"1936dcb1-1a31-4466-a3e7-c65e6dada324\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk"
Jul 28 16:02:13.302998 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:13.302970    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1936dcb1-1a31-4466-a3e7-c65e6dada324-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-2gmnk\" (UID: \"1936dcb1-1a31-4466-a3e7-c65e6dada324\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk"
Jul 28 16:02:13.584571 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:13.584471    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk"
Jul 28 16:02:14.263712 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:14.263673    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk"]
Jul 28 16:02:14.681048 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:14.681011    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk" event={"ID":"1936dcb1-1a31-4466-a3e7-c65e6dada324","Type":"ContainerStarted","Data":"824d7b9e2c82dcf05b57d706764444d8f83f3904e294ffb86db733c917521d46"}
Jul 28 16:02:14.682349 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:14.682323    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jdnzr" event={"ID":"992fecb8-694b-4cf6-ae74-53488252c696","Type":"ContainerStarted","Data":"903581220ea4136d70d2c3a691fbc3e49bf301a3d84d7c8cf20d61b40835764e"}
Jul 28 16:02:14.683914 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:14.683889    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l8xg8" event={"ID":"157a700c-ac88-481d-a050-7b2e3c0434e9","Type":"ContainerStarted","Data":"f1dc1070704553bb4ca2fc59402935569bd2982efbb9d2f098fdf0a145bfb044"}
Jul 28 16:02:14.684140 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:14.683930    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l8xg8" event={"ID":"157a700c-ac88-481d-a050-7b2e3c0434e9","Type":"ContainerStarted","Data":"d7d177463d70a2a2275064a7b48a33a0e0c1ccb4b34424a0ff8f9b1e9f5cd6e2"}
Jul 28 16:02:14.684140 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:14.684023    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:14.685284 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:14.685258    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-scthp" event={"ID":"60836054-f2ad-4604-a028-77b976a9cf5b","Type":"ContainerStarted","Data":"44404ad98104395d0215d5ee171e3924c455acd64099c84a5d031249c95d88a8"}
Jul 28 16:02:14.703447 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:14.703395    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-l8xg8" podStartSLOduration=1.775261345 podStartE2EDuration="7.703377931s" totalImagesPullingTime="5.928116586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:14.702636521 +0000 UTC m=+60.847015075" watchObservedRunningTime="2026-07-28 16:02:14.703377931 +0000 UTC m=+60.847756475"
Jul 28 16:02:14.718351 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:14.718260    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-scthp" podStartSLOduration=1.813125087 podStartE2EDuration="7.718237791s" totalImagesPullingTime="5.905112704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:14.716991109 +0000 UTC m=+60.861369663" watchObservedRunningTime="2026-07-28 16:02:14.718237791 +0000 UTC m=+60.862616344"
Jul 28 16:02:15.622646 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:15.622613    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hz6tj"
Jul 28 16:02:15.691348 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:15.691309    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk" event={"ID":"1936dcb1-1a31-4466-a3e7-c65e6dada324","Type":"ContainerStarted","Data":"f2abebe774d84d1eabae036690db3c176037e1107631363fd37f3681e891dabd"}
Jul 28 16:02:15.711440 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:15.711374    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk" podStartSLOduration=2.403939245 podStartE2EDuration="3.711351575s" totalImagesPullingTime="1.30741233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:15.710921149 +0000 UTC m=+61.855299729" watchObservedRunningTime="2026-07-28 16:02:15.711351575 +0000 UTC m=+61.855730129"
Jul 28 16:02:16.695032 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:16.694985    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk"
Jul 28 16:02:16.700997 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:16.700965    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-2gmnk"
Jul 28 16:02:18.703805 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:18.703722    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jdnzr" event={"ID":"992fecb8-694b-4cf6-ae74-53488252c696","Type":"ContainerStarted","Data":"967dc8389fa528a8930fd9b96acac5dc7a93b655624fad0dd0777ff8d72b4bbb"}
Jul 28 16:02:19.050340 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.050304    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:02:19.053071 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.053041    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 16:02:19.063783 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.063745    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b549f8a9-6b3c-42bc-bbc8-11d57f9bab80-metrics-certs\") pod \"network-metrics-daemon-f2jjj\" (UID: \"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80\") " pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:02:19.150786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.150746    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:02:19.150982 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.150838    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpv9t\" (UniqueName: \"kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t\") pod \"network-check-target-qknsz\" (UID: \"bd6f7fa4-5752-4b05-a359-9277b293c7b6\") " pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:02:19.153620 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.153587    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 16:02:19.153620 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.153587    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 16:02:19.163631 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.163398    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 16:02:19.163984 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.163950    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d700d5d0-afa0-407f-b47a-768e795a3949-original-pull-secret\") pod \"global-pull-secret-syncer-9fplh\" (UID: \"d700d5d0-afa0-407f-b47a-768e795a3949\") " pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:02:19.175240 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.175205    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpv9t\" (UniqueName: \"kubernetes.io/projected/bd6f7fa4-5752-4b05-a359-9277b293c7b6-kube-api-access-vpv9t\") pod \"network-check-target-qknsz\" (UID: \"bd6f7fa4-5752-4b05-a359-9277b293c7b6\") " pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:02:19.253754 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.253722    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-84r49\""
Jul 28 16:02:19.257538 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.257505    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9fplh"
Jul 28 16:02:19.262045 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.262013    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:02:19.265490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.265463    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kkdj2\""
Jul 28 16:02:19.274256 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.273980    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2jjj"
Jul 28 16:02:19.419174 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.418843    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9fplh"]
Jul 28 16:02:19.424902 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:19.424849    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd700d5d0_afa0_407f_b47a_768e795a3949.slice/crio-383ae994f9c598d77778a40302fd8bd5b6301a6a1ee77e1315c8932921b2fd64 WatchSource:0}: Error finding container 383ae994f9c598d77778a40302fd8bd5b6301a6a1ee77e1315c8932921b2fd64: Status 404 returned error can't find the container with id 383ae994f9c598d77778a40302fd8bd5b6301a6a1ee77e1315c8932921b2fd64
Jul 28 16:02:19.449140 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.449088    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qknsz"]
Jul 28 16:02:19.453039 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:19.452996    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd6f7fa4_5752_4b05_a359_9277b293c7b6.slice/crio-d4a4e20e674169017d443318e2d5172b6e60eed656361bf8ba9aaeb41c613a08 WatchSource:0}: Error finding container d4a4e20e674169017d443318e2d5172b6e60eed656361bf8ba9aaeb41c613a08: Status 404 returned error can't find the container with id d4a4e20e674169017d443318e2d5172b6e60eed656361bf8ba9aaeb41c613a08
Jul 28 16:02:19.466376 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.466290    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f2jjj"]
Jul 28 16:02:19.471311 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:19.471280    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb549f8a9_6b3c_42bc_bbc8_11d57f9bab80.slice/crio-a95cf6dc2fd0e31b091489ef819e36f0b1a52edd4ba4e014ea16d3f58ec4cc0c WatchSource:0}: Error finding container a95cf6dc2fd0e31b091489ef819e36f0b1a52edd4ba4e014ea16d3f58ec4cc0c: Status 404 returned error can't find the container with id a95cf6dc2fd0e31b091489ef819e36f0b1a52edd4ba4e014ea16d3f58ec4cc0c
Jul 28 16:02:19.709141 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.709035    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f2jjj" event={"ID":"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80","Type":"ContainerStarted","Data":"a95cf6dc2fd0e31b091489ef819e36f0b1a52edd4ba4e014ea16d3f58ec4cc0c"}
Jul 28 16:02:19.710448 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.710417    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qknsz" event={"ID":"bd6f7fa4-5752-4b05-a359-9277b293c7b6","Type":"ContainerStarted","Data":"d4a4e20e674169017d443318e2d5172b6e60eed656361bf8ba9aaeb41c613a08"}
Jul 28 16:02:19.711632 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:19.711610    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9fplh" event={"ID":"d700d5d0-afa0-407f-b47a-768e795a3949","Type":"ContainerStarted","Data":"383ae994f9c598d77778a40302fd8bd5b6301a6a1ee77e1315c8932921b2fd64"}
Jul 28 16:02:22.726279 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.726239    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jdnzr" event={"ID":"992fecb8-694b-4cf6-ae74-53488252c696","Type":"ContainerStarted","Data":"0731740b70500dc55a86c9abcd02eba6428f5d33bc5ef8c111c501023bda948f"}
Jul 28 16:02:22.728273 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.728231    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f2jjj" event={"ID":"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80","Type":"ContainerStarted","Data":"bcc152b9cab425a15eeb35b93caee24ac1bc74e0245a9d3b2df239f3309f342d"}
Jul 28 16:02:22.728429 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.728296    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f2jjj" event={"ID":"b549f8a9-6b3c-42bc-bbc8-11d57f9bab80","Type":"ContainerStarted","Data":"c8a389ccc0dbe98ed898e3b5328ad6baaa45ed4a0318a02014a7bfaca30f7d91"}
Jul 28 16:02:22.745783 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.745720    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jdnzr" podStartSLOduration=2.401961308 podStartE2EDuration="15.745700549s" totalImagesPullingTime="13.343739241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:22.744317066 +0000 UTC m=+68.888695620" watchObservedRunningTime="2026-07-28 16:02:22.745700549 +0000 UTC m=+68.890079104"
Jul 28 16:02:22.761924 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.761862    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-f2jjj" podStartSLOduration=66.45750346 podStartE2EDuration="1m8.76184014s" totalImagesPullingTime="2.30433668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:22.761194709 +0000 UTC m=+68.905573265" watchObservedRunningTime="2026-07-28 16:02:22.76184014 +0000 UTC m=+68.906218733"
Jul 28 16:02:22.938044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.938005    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-675f96fd95-mk6xb"]
Jul 28 16:02:22.953058 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.953030    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:22.954442 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.954417    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-675f96fd95-mk6xb"]
Jul 28 16:02:22.956668 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.956638    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9vttv\""
Jul 28 16:02:22.956825 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.956667    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 16:02:22.957217 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.957197    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 16:02:22.957368 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.957312    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 16:02:22.957542 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.957373    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 16:02:22.957542 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.957385    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 16:02:22.957542 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.957491    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 16:02:22.957800 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.957784    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 16:02:22.982392 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.982291    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc1726b6-c780-4990-b864-a8be438c12f7-console-oauth-config\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:22.982392 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.982333    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-console-config\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:22.982392 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.982359    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcpb6\" (UniqueName: \"kubernetes.io/projected/cc1726b6-c780-4990-b864-a8be438c12f7-kube-api-access-gcpb6\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:22.982658 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.982437    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-service-ca\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:22.982658 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.982456    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-oauth-serving-cert\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:22.982658 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:22.982497    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc1726b6-c780-4990-b864-a8be438c12f7-console-serving-cert\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.083651 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.083603    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-service-ca\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.083651 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.083650    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-oauth-serving-cert\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.083860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.083691    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc1726b6-c780-4990-b864-a8be438c12f7-console-serving-cert\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.083860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.083738    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc1726b6-c780-4990-b864-a8be438c12f7-console-oauth-config\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.083860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.083763    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-console-config\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.083860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.083777    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcpb6\" (UniqueName: \"kubernetes.io/projected/cc1726b6-c780-4990-b864-a8be438c12f7-kube-api-access-gcpb6\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.084526 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.084449    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-oauth-serving-cert\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.084526 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.084470    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-service-ca\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.084776 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.084706    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-console-config\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.087145 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.087116    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc1726b6-c780-4990-b864-a8be438c12f7-console-oauth-config\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.087145 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.087135    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc1726b6-c780-4990-b864-a8be438c12f7-console-serving-cert\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.101529 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.101491    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcpb6\" (UniqueName: \"kubernetes.io/projected/cc1726b6-c780-4990-b864-a8be438c12f7-kube-api-access-gcpb6\") pod \"console-675f96fd95-mk6xb\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") " pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.263063 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.262964    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:23.264796 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.264763    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wd94m"]
Jul 28 16:02:23.305403 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.305375    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.307736 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.307708    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 16:02:23.307889 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.307795    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 16:02:23.307889 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.307708    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vkdgx\""
Jul 28 16:02:23.308154 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.308134    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 16:02:23.308274 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.308190    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 16:02:23.308274 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.308195    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 16:02:23.308274 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.308258    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 16:02:23.386297 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.386260    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-accelerators-collector-config\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.386297 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.386304    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-tls\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.386548 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.386408    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-textfile\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.386548 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.386435    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-root\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.386548 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.386463    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.386548 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.386530    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-wtmp\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.386847 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.386566    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nk59\" (UniqueName: \"kubernetes.io/projected/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-kube-api-access-7nk59\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.386847 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.386603    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-metrics-client-ca\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.386847 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.386649    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-sys\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.487577 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.487540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-root\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.487786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.487590    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.487786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.487621    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-wtmp\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.487786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.487640    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nk59\" (UniqueName: \"kubernetes.io/projected/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-kube-api-access-7nk59\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.487786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.487656    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-metrics-client-ca\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.487786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.487660    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-root\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.487786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.487695    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-sys\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.487786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.487724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-accelerators-collector-config\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.487786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.487752    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-tls\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.488228 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.487806    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-textfile\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.488228 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.487818    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-wtmp\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.488228 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.487873    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-sys\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.488228 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.488193    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-textfile\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.488482 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.488460    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-accelerators-collector-config\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.488670 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.488627    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-metrics-client-ca\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.491194 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.491144    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-tls\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.491194 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.491166    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.500630 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.500600    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nk59\" (UniqueName: \"kubernetes.io/projected/12b4dc4d-d95c-410b-9e4d-df1e16ecbda7-kube-api-access-7nk59\") pod \"node-exporter-wd94m\" (UID: \"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7\") " pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.617450 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:23.617354    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wd94m"
Jul 28 16:02:23.956553 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:23.956522    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12b4dc4d_d95c_410b_9e4d_df1e16ecbda7.slice/crio-b7054048b542d38025f7dd1f8cd1a0d3a1dd7ebf02fcbde0ce9cfea889ab411a WatchSource:0}: Error finding container b7054048b542d38025f7dd1f8cd1a0d3a1dd7ebf02fcbde0ce9cfea889ab411a: Status 404 returned error can't find the container with id b7054048b542d38025f7dd1f8cd1a0d3a1dd7ebf02fcbde0ce9cfea889ab411a
Jul 28 16:02:24.119353 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.119276    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-675f96fd95-mk6xb"]
Jul 28 16:02:24.122363 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:24.122327    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc1726b6_c780_4990_b864_a8be438c12f7.slice/crio-1034842a4ed682fdf59b28b9426cb94bcc87202f26a3c952113ea9312a3b6edc WatchSource:0}: Error finding container 1034842a4ed682fdf59b28b9426cb94bcc87202f26a3c952113ea9312a3b6edc: Status 404 returned error can't find the container with id 1034842a4ed682fdf59b28b9426cb94bcc87202f26a3c952113ea9312a3b6edc
Jul 28 16:02:24.341532 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.341488    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 16:02:24.344716 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.344689    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.350156 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.348182    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hq6rk\""
Jul 28 16:02:24.350156 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.348310    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 16:02:24.350156 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.348466    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 16:02:24.350156 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.348532    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 16:02:24.350156 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.348718    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 16:02:24.350156 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.348773    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 16:02:24.350156 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.349722    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 16:02:24.350156 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.350036    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 16:02:24.350667 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.350305    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 16:02:24.350667 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.350326    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 16:02:24.364702 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.364670    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 16:02:24.396352 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396312    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.396352 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396365    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.396625 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396401    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.396625 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396515    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-config-volume\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.396625 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396565    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcmkr\" (UniqueName: \"kubernetes.io/projected/21031462-b95c-4f26-9e14-d8f115b0d6a7-kube-api-access-vcmkr\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.396792 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396691    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.396792 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396715    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.396792 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396746    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.396792 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396774    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-web-config\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.397006 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396803    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.397006 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396835    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.397006 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396917    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21031462-b95c-4f26-9e14-d8f115b0d6a7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.397006 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.396955    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21031462-b95c-4f26-9e14-d8f115b0d6a7-config-out\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.498297 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498259    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.498569 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498311    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.498569 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498353    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-config-volume\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.498569 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498392    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcmkr\" (UniqueName: \"kubernetes.io/projected/21031462-b95c-4f26-9e14-d8f115b0d6a7-kube-api-access-vcmkr\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.498569 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.498569 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498450    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.498569 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498480    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.498569 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498507    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-web-config\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.498569 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498535    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.498569 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498554    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.499035 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498582    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21031462-b95c-4f26-9e14-d8f115b0d6a7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.499035 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498604    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21031462-b95c-4f26-9e14-d8f115b0d6a7-config-out\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.499035 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.498629    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.499035 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:24.498964    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-trusted-ca-bundle podName:21031462-b95c-4f26-9e14-d8f115b0d6a7 nodeName:}" failed. No retries permitted until 2026-07-28 16:02:24.998939452 +0000 UTC m=+71.143317988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7") : configmap references non-existent config key: ca-bundle.crt
Jul 28 16:02:24.499301 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.499277    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.500166 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.499964    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.500511 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:24.500488    2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 28 16:02:24.500617 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:02:24.500568    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-main-tls podName:21031462-b95c-4f26-9e14-d8f115b0d6a7 nodeName:}" failed. No retries permitted until 2026-07-28 16:02:25.000549822 +0000 UTC m=+71.144928360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7") : secret "alertmanager-main-tls" not found
Jul 28 16:02:24.502690 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.502663    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-config-volume\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.502819 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.502707    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21031462-b95c-4f26-9e14-d8f115b0d6a7-config-out\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.503547 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.503164    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.503547 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.503496    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.504364 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.504314    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.504848 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.504815    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.505314 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.505291    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-web-config\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.505799 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.505758    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21031462-b95c-4f26-9e14-d8f115b0d6a7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.511547 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.511463    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcmkr\" (UniqueName: \"kubernetes.io/projected/21031462-b95c-4f26-9e14-d8f115b0d6a7-kube-api-access-vcmkr\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:24.694645 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.694549    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l8xg8"
Jul 28 16:02:24.744216 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.744178    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qknsz" event={"ID":"bd6f7fa4-5752-4b05-a359-9277b293c7b6","Type":"ContainerStarted","Data":"e3d8c97b17708ba27020f57a8152277db8a6c4abb29207d8e7abe73ac0cd0ff8"}
Jul 28 16:02:24.744469 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.744448    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:02:24.750825 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.750721    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wd94m" event={"ID":"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7","Type":"ContainerStarted","Data":"b7054048b542d38025f7dd1f8cd1a0d3a1dd7ebf02fcbde0ce9cfea889ab411a"}
Jul 28 16:02:24.754026 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.753944    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675f96fd95-mk6xb" event={"ID":"cc1726b6-c780-4990-b864-a8be438c12f7","Type":"ContainerStarted","Data":"1034842a4ed682fdf59b28b9426cb94bcc87202f26a3c952113ea9312a3b6edc"}
Jul 28 16:02:24.761463 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:24.760326    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qknsz" podStartSLOduration=66.233105865 podStartE2EDuration="1m10.760304354s" totalImagesPullingTime="4.527198489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:24.759547378 +0000 UTC m=+70.903925932" watchObservedRunningTime="2026-07-28 16:02:24.760304354 +0000 UTC m=+70.904682907"
Jul 28 16:02:25.003639 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:25.003597    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:25.004166 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:25.003659    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:25.004868 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:25.004698    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:25.012085 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:25.012021    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:25.259992 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:25.259958    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:02:25.466700 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:25.466659    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 16:02:25.489401 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:25.489366    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21031462_b95c_4f26_9e14_d8f115b0d6a7.slice/crio-13acec3635eae47455566be783c0017456fba3599c3a77174c6d223a1aee46b7 WatchSource:0}: Error finding container 13acec3635eae47455566be783c0017456fba3599c3a77174c6d223a1aee46b7: Status 404 returned error can't find the container with id 13acec3635eae47455566be783c0017456fba3599c3a77174c6d223a1aee46b7
Jul 28 16:02:25.761487 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:25.761438    2575 generic.go:356] "Generic (PLEG): container finished" podID="12b4dc4d-d95c-410b-9e4d-df1e16ecbda7" containerID="9e71959f93b255c56806457b2e759181eb244fd9853d4aa26adf64853839476f" exitCode=0
Jul 28 16:02:25.761688 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:25.761541    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wd94m" event={"ID":"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7","Type":"ContainerDied","Data":"9e71959f93b255c56806457b2e759181eb244fd9853d4aa26adf64853839476f"}
Jul 28 16:02:25.763487 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:25.763461    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerStarted","Data":"13acec3635eae47455566be783c0017456fba3599c3a77174c6d223a1aee46b7"}
Jul 28 16:02:26.769321 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:26.769275    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wd94m" event={"ID":"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7","Type":"ContainerStarted","Data":"063faa0b56e03c59ef11bff43a112ee7aad5c61c0fb28548ca28d5d7188b5cd0"}
Jul 28 16:02:26.769806 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:26.769351    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wd94m" event={"ID":"12b4dc4d-d95c-410b-9e4d-df1e16ecbda7","Type":"ContainerStarted","Data":"f2d3631d1d2d7043e71f0612428619209ffcf78ec3be5c87eca05164fc019e2a"}
Jul 28 16:02:26.791838 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:26.791769    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wd94m" podStartSLOduration=2.423444344 podStartE2EDuration="3.791750015s" totalImagesPullingTime="1.233117893s" totalInitContainerRuntime="135.187778ms" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:26.790339008 +0000 UTC m=+72.934717617" watchObservedRunningTime="2026-07-28 16:02:26.791750015 +0000 UTC m=+72.936128567"
Jul 28 16:02:27.480744 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.480708    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"]
Jul 28 16:02:27.483172 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.483140    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.486420 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.486361    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 16:02:27.486420 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.486372    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 16:02:27.486420 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.486386    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 16:02:27.486801 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.486423    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-v6zjj\""
Jul 28 16:02:27.486801 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.486447    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3rsek6d4pjh41\""
Jul 28 16:02:27.486801 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.486377    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 16:02:27.492892 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.492864    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"]
Jul 28 16:02:27.527584 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.527547    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c11ce7a4-345c-4fe3-8fce-e240f1415048-client-ca-bundle\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.527584 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.527597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c11ce7a4-345c-4fe3-8fce-e240f1415048-secret-metrics-server-client-certs\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.527830 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.527631    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j22gs\" (UniqueName: \"kubernetes.io/projected/c11ce7a4-345c-4fe3-8fce-e240f1415048-kube-api-access-j22gs\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.527830 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.527740    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c11ce7a4-345c-4fe3-8fce-e240f1415048-metrics-server-audit-profiles\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.527830 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.527798    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c11ce7a4-345c-4fe3-8fce-e240f1415048-secret-metrics-server-tls\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.527951 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.527870    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c11ce7a4-345c-4fe3-8fce-e240f1415048-audit-log\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.527951 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.527924    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c11ce7a4-345c-4fe3-8fce-e240f1415048-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.628970 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.628924    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c11ce7a4-345c-4fe3-8fce-e240f1415048-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.629179 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.628982    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c11ce7a4-345c-4fe3-8fce-e240f1415048-client-ca-bundle\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.629179 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.629016    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c11ce7a4-345c-4fe3-8fce-e240f1415048-secret-metrics-server-client-certs\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.629179 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.629037    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j22gs\" (UniqueName: \"kubernetes.io/projected/c11ce7a4-345c-4fe3-8fce-e240f1415048-kube-api-access-j22gs\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.629179 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.629077    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c11ce7a4-345c-4fe3-8fce-e240f1415048-metrics-server-audit-profiles\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.629179 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.629130    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c11ce7a4-345c-4fe3-8fce-e240f1415048-secret-metrics-server-tls\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.629179 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.629161    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c11ce7a4-345c-4fe3-8fce-e240f1415048-audit-log\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.629808 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.629753    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c11ce7a4-345c-4fe3-8fce-e240f1415048-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.629930 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.629835    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c11ce7a4-345c-4fe3-8fce-e240f1415048-audit-log\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.630563 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.630538    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c11ce7a4-345c-4fe3-8fce-e240f1415048-metrics-server-audit-profiles\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.632110 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.632069    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c11ce7a4-345c-4fe3-8fce-e240f1415048-client-ca-bundle\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.632209 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.632169    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c11ce7a4-345c-4fe3-8fce-e240f1415048-secret-metrics-server-tls\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.632209 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.632204    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c11ce7a4-345c-4fe3-8fce-e240f1415048-secret-metrics-server-client-certs\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.640323 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.640290    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j22gs\" (UniqueName: \"kubernetes.io/projected/c11ce7a4-345c-4fe3-8fce-e240f1415048-kube-api-access-j22gs\") pod \"metrics-server-7c66fdcb8f-vk9c9\" (UID: \"c11ce7a4-345c-4fe3-8fce-e240f1415048\") " pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:27.794883 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:27.794791    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:28.013365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.013319    2575 patch_prober.go:28] interesting pod/image-registry-84b869bf79-7dm68 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 16:02:28.013561 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.013387    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-84b869bf79-7dm68" podUID="9cef1e5c-6531-4245-9341-c6c83e315dbe" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:02:28.386392 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.386352    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8574776b9-85vlk"]
Jul 28 16:02:28.388779 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.388757    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.391849 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.391815    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 16:02:28.392213 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.392188    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 16:02:28.392403 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.392216    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 16:02:28.392403 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.392191    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 16:02:28.392633 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.392613    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 16:02:28.392758 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.392615    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9d6nx\""
Jul 28 16:02:28.400350 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.400312    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 16:02:28.406039 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.405997    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8574776b9-85vlk"]
Jul 28 16:02:28.437086 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.437045    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81524ca0-40e5-4835-acd1-abe87263a284-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.437271 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.437130    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81524ca0-40e5-4835-acd1-abe87263a284-metrics-client-ca\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.437271 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.437187    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81524ca0-40e5-4835-acd1-abe87263a284-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.437271 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.437243    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/81524ca0-40e5-4835-acd1-abe87263a284-secret-telemeter-client\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.437420 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.437289    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/81524ca0-40e5-4835-acd1-abe87263a284-federate-client-tls\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.437420 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.437319    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/81524ca0-40e5-4835-acd1-abe87263a284-telemeter-client-tls\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.437420 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.437347    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndvgd\" (UniqueName: \"kubernetes.io/projected/81524ca0-40e5-4835-acd1-abe87263a284-kube-api-access-ndvgd\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.437420 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.437374    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81524ca0-40e5-4835-acd1-abe87263a284-serving-certs-ca-bundle\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.538588 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.538541    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81524ca0-40e5-4835-acd1-abe87263a284-metrics-client-ca\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.538798 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.538632    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81524ca0-40e5-4835-acd1-abe87263a284-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.538798 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.538670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/81524ca0-40e5-4835-acd1-abe87263a284-secret-telemeter-client\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.538798 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.538706    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/81524ca0-40e5-4835-acd1-abe87263a284-federate-client-tls\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.538798 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.538734    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/81524ca0-40e5-4835-acd1-abe87263a284-telemeter-client-tls\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.538798 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.538759    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndvgd\" (UniqueName: \"kubernetes.io/projected/81524ca0-40e5-4835-acd1-abe87263a284-kube-api-access-ndvgd\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.538798 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.538786    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81524ca0-40e5-4835-acd1-abe87263a284-serving-certs-ca-bundle\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.539121 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.539038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81524ca0-40e5-4835-acd1-abe87263a284-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.539496 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.539448    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81524ca0-40e5-4835-acd1-abe87263a284-metrics-client-ca\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.539620 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.539494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81524ca0-40e5-4835-acd1-abe87263a284-serving-certs-ca-bundle\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.539877 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.539849    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81524ca0-40e5-4835-acd1-abe87263a284-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.541973 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.541943    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81524ca0-40e5-4835-acd1-abe87263a284-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.542083 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.542042    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/81524ca0-40e5-4835-acd1-abe87263a284-federate-client-tls\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.542252 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.542228    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/81524ca0-40e5-4835-acd1-abe87263a284-telemeter-client-tls\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.542354 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.542235    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/81524ca0-40e5-4835-acd1-abe87263a284-secret-telemeter-client\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.548969 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.548925    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndvgd\" (UniqueName: \"kubernetes.io/projected/81524ca0-40e5-4835-acd1-abe87263a284-kube-api-access-ndvgd\") pod \"telemeter-client-8574776b9-85vlk\" (UID: \"81524ca0-40e5-4835-acd1-abe87263a284\") " pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:28.705649 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:28.705556    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8574776b9-85vlk"
Jul 28 16:02:29.277776 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.277739    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"]
Jul 28 16:02:29.301228 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:29.301191    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc11ce7a4_345c_4fe3_8fce_e240f1415048.slice/crio-daa26f59baf4502cff5269c03b221819aa80c1f35327d471083ba39e99c54a76 WatchSource:0}: Error finding container daa26f59baf4502cff5269c03b221819aa80c1f35327d471083ba39e99c54a76: Status 404 returned error can't find the container with id daa26f59baf4502cff5269c03b221819aa80c1f35327d471083ba39e99c54a76
Jul 28 16:02:29.302896 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.302867    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8574776b9-85vlk"]
Jul 28 16:02:29.306177 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:29.306146    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81524ca0_40e5_4835_acd1_abe87263a284.slice/crio-5ca77063835ab4de170b0ee239781d6ea0f0cc0bd440c9f0d224cd0674bfd9c9 WatchSource:0}: Error finding container 5ca77063835ab4de170b0ee239781d6ea0f0cc0bd440c9f0d224cd0674bfd9c9: Status 404 returned error can't find the container with id 5ca77063835ab4de170b0ee239781d6ea0f0cc0bd440c9f0d224cd0674bfd9c9
Jul 28 16:02:29.512044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.511999    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 16:02:29.515879 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.515858    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.521081 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521044    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 16:02:29.521274 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521142    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 16:02:29.521274 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521160    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 16:02:29.521274 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521188    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 16:02:29.521274 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521212    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 16:02:29.521274 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521232    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 16:02:29.521274 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521194    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 16:02:29.521274 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521149    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 16:02:29.521753 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521626    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 16:02:29.521753 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521636    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 16:02:29.521753 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521643    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 16:02:29.521753 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521636    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 16:02:29.522056 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.521868    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5iuqekapmmjgl\""
Jul 28 16:02:29.522206 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.522055    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2txww\""
Jul 28 16:02:29.527422 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.527401    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 16:02:29.528926 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.528910    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 16:02:29.542968 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.542805    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 16:02:29.649030 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.648988    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-web-config\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649030 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649028    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649321 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649048    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649321 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649068    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649321 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649176    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649321 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649222    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649321 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649251    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-config-out\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649321 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649274    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649321 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649309    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-config\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649541 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649335    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649541 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649369    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649541 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649429    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649541 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649459    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649541 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649477    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649541 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649497    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649541 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649514    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649541 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649538    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649796 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649581    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649796 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649610    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p97rn\" (UniqueName: \"kubernetes.io/projected/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-kube-api-access-p97rn\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.649796 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.649639    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.671793 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.671760    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84b869bf79-7dm68"
Jul 28 16:02:29.750625 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.750580    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.750810 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.750653    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p97rn\" (UniqueName: \"kubernetes.io/projected/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-kube-api-access-p97rn\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.750810 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.750695    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751029 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751004    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-web-config\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751086 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751057    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751151 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751127    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751188    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751314 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751220    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751314 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751253    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-config-out\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751314 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751279    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751460 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751317    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-config\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751460 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751319    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751460 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751365    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751460 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751407    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751758 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751738    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.751914 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.751888    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.753621 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.753589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.753743 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.753633    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.753743 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.753663    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.753743 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.753703    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.753743 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.753731    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.753962 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.753771    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.754016 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.753969    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.754482 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.754125    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-web-config\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.754482 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.754170    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.754656 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.754596    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.754656 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.754616    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-config-out\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.754767 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.754740    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.755200 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.755049    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.755200 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.755046    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.755200 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.755084    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.755200 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.755085    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.755707 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.755663    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.755891 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.755869    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.756770 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.756751    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.756927 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.756910    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.757164 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.757143    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-config\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.757396 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.757379    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.762202 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.762155    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p97rn\" (UniqueName: \"kubernetes.io/projected/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-kube-api-access-p97rn\") pod \"prometheus-k8s-0\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.782111 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.782062    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9" event={"ID":"c11ce7a4-345c-4fe3-8fce-e240f1415048","Type":"ContainerStarted","Data":"daa26f59baf4502cff5269c03b221819aa80c1f35327d471083ba39e99c54a76"}
Jul 28 16:02:29.783241 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.783214    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8574776b9-85vlk" event={"ID":"81524ca0-40e5-4835-acd1-abe87263a284","Type":"ContainerStarted","Data":"5ca77063835ab4de170b0ee239781d6ea0f0cc0bd440c9f0d224cd0674bfd9c9"}
Jul 28 16:02:29.784653 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.784628    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9fplh" event={"ID":"d700d5d0-afa0-407f-b47a-768e795a3949","Type":"ContainerStarted","Data":"d35749a223057030f99333ca25c3c006ea2ae7248272d08b5fc0e7473908d017"}
Jul 28 16:02:29.786124 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.786082    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675f96fd95-mk6xb" event={"ID":"cc1726b6-c780-4990-b864-a8be438c12f7","Type":"ContainerStarted","Data":"248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36"}
Jul 28 16:02:29.787491 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.787462    2575 generic.go:356] "Generic (PLEG): container finished" podID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerID="c9fd8a65b1ceb73a5a201a15b43ba6f280dc1f3e48a7d8e1b1f798dbfe0d7031" exitCode=0
Jul 28 16:02:29.787592 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.787514    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerDied","Data":"c9fd8a65b1ceb73a5a201a15b43ba6f280dc1f3e48a7d8e1b1f798dbfe0d7031"}
Jul 28 16:02:29.822598 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.822534    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9fplh" podStartSLOduration=66.13088556 podStartE2EDuration="1m15.822510716s" totalImagesPullingTime="9.691625156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:29.801347949 +0000 UTC m=+75.945726516" watchObservedRunningTime="2026-07-28 16:02:29.822510716 +0000 UTC m=+75.966889270"
Jul 28 16:02:29.823721 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.823676    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-675f96fd95-mk6xb" podStartSLOduration=2.8377882899999998 podStartE2EDuration="7.823657293s" totalImagesPullingTime="4.985869003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:29.822982017 +0000 UTC m=+75.967360565" watchObservedRunningTime="2026-07-28 16:02:29.823657293 +0000 UTC m=+75.968035846"
Jul 28 16:02:29.825532 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.825506    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:29.995803 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:29.995712    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 16:02:30.000663 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:02:30.000610    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e3e8fcd_e8dc_42e1_a23a_3fe6ae7b2fc9.slice/crio-1c7c3c0ccae24c466aca517f86ba129064449c3f75ed79fbf5ff3572d235f4a6 WatchSource:0}: Error finding container 1c7c3c0ccae24c466aca517f86ba129064449c3f75ed79fbf5ff3572d235f4a6: Status 404 returned error can't find the container with id 1c7c3c0ccae24c466aca517f86ba129064449c3f75ed79fbf5ff3572d235f4a6
Jul 28 16:02:30.798265 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:30.798234    2575 generic.go:356] "Generic (PLEG): container finished" podID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerID="59b5ccbabdf82efbf6fee5ebfa89e947e09a2c91d549c1084a911eb483a970d0" exitCode=0
Jul 28 16:02:30.798695 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:30.798294    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerDied","Data":"59b5ccbabdf82efbf6fee5ebfa89e947e09a2c91d549c1084a911eb483a970d0"}
Jul 28 16:02:30.798695 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:30.798377    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerStarted","Data":"1c7c3c0ccae24c466aca517f86ba129064449c3f75ed79fbf5ff3572d235f4a6"}
Jul 28 16:02:31.803540 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:31.803493    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9" event={"ID":"c11ce7a4-345c-4fe3-8fce-e240f1415048","Type":"ContainerStarted","Data":"d997bee7823afe6d7d0c23989624b2703a22aaab7721d80a2d0b44420ae39c04"}
Jul 28 16:02:31.805503 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:31.805472    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8574776b9-85vlk" event={"ID":"81524ca0-40e5-4835-acd1-abe87263a284","Type":"ContainerStarted","Data":"71cd32036b6caa1706c39a58e9479c83f9c74b994f95d58dbdd94b2cc2b7dd35"}
Jul 28 16:02:31.805631 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:31.805537    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8574776b9-85vlk" event={"ID":"81524ca0-40e5-4835-acd1-abe87263a284","Type":"ContainerStarted","Data":"8d5043273015e65736874a342b9ded8336b4d2be81f0c30c56493111343cb49b"}
Jul 28 16:02:31.805631 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:31.805570    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8574776b9-85vlk" event={"ID":"81524ca0-40e5-4835-acd1-abe87263a284","Type":"ContainerStarted","Data":"35225424686c5d3477e75124b40d1157ba6a8a90753a7122b04ec6e300cd2b3f"}
Jul 28 16:02:31.823646 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:31.823594    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9" podStartSLOduration=3.406834788 podStartE2EDuration="4.823576373s" totalImagesPullingTime="1.416741585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:31.82227259 +0000 UTC m=+77.966651144" watchObservedRunningTime="2026-07-28 16:02:31.823576373 +0000 UTC m=+77.967954990"
Jul 28 16:02:31.846727 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:31.846522    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8574776b9-85vlk" podStartSLOduration=1.726001012 podStartE2EDuration="3.846502942s" totalImagesPullingTime="2.12050193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:31.845484431 +0000 UTC m=+77.989862983" watchObservedRunningTime="2026-07-28 16:02:31.846502942 +0000 UTC m=+77.990881495"
Jul 28 16:02:33.264132 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:33.264072    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:33.264612 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:33.264184    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:33.270759 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:33.270723    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:33.818668 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:33.818574    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerStarted","Data":"20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541"}
Jul 28 16:02:33.818668 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:33.818664    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerStarted","Data":"5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa"}
Jul 28 16:02:33.818871 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:33.818708    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerStarted","Data":"2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791"}
Jul 28 16:02:33.818871 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:33.818744    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerStarted","Data":"36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024"}
Jul 28 16:02:33.818871 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:33.818775    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerStarted","Data":"b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916"}
Jul 28 16:02:33.823188 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:33.823159    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:02:34.826299 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:34.826261    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerStarted","Data":"fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29"}
Jul 28 16:02:34.867022 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:34.866915    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.724347099 podStartE2EDuration="10.866893085s" totalImagesPullingTime="8.121706524s" totalInitContainerRuntime="20.839462ms" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:34.860990262 +0000 UTC m=+81.005368825" watchObservedRunningTime="2026-07-28 16:02:34.866893085 +0000 UTC m=+81.011271638"
Jul 28 16:02:36.838501 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:36.838463    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerStarted","Data":"89a961e00275ad769e1825eed8eb860c7598b3f9da2463ae8c6d1cf07167ffba"}
Jul 28 16:02:36.838868 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:36.838533    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerStarted","Data":"c5e2ebf3c76eb9806360e9a2c5ee78ae38b3aac688744680f205032dfcca99e9"}
Jul 28 16:02:38.849594 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:38.849555    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerStarted","Data":"31cd7f92c64710d9049c1319f45ee7d670b94321679e5f688fd8d80c66f06895"}
Jul 28 16:02:38.849994 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:38.849630    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerStarted","Data":"df6df453b7d30160c4bd00e2d7d30b017ef47b33c0214cde50b522a390909cf7"}
Jul 28 16:02:38.849994 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:38.849658    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerStarted","Data":"f9b91832936784c3fa0272b995feb6a764cf20e44809d71a828b46609403d1c9"}
Jul 28 16:02:38.849994 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:38.849682    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerStarted","Data":"cebf1efb08e6035cc41ea4e5db82a64584809a59e6239bf847b112d70eee4d94"}
Jul 28 16:02:38.915912 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:38.915827    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.551230657 podStartE2EDuration="9.915809757s" totalImagesPullingTime="7.339143651s" totalInitContainerRuntime="25.435449ms" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:38.909767634 +0000 UTC m=+85.054146187" watchObservedRunningTime="2026-07-28 16:02:38.915809757 +0000 UTC m=+85.060188310"
Jul 28 16:02:39.826254 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:39.826210    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:02:47.795917 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:47.795870    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:47.796412 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:47.795942    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:02:51.039290 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:51.039253    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-675f96fd95-mk6xb"]
Jul 28 16:02:55.766113 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:02:55.766067    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qknsz"
Jul 28 16:03:07.801063 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:07.801029    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:03:07.805019 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:07.804994    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7c66fdcb8f-vk9c9"
Jul 28 16:03:16.059943 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.059865    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-675f96fd95-mk6xb" podUID="cc1726b6-c780-4990-b864-a8be438c12f7" containerName="console" containerID="cri-o://248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36" gracePeriod=15
Jul 28 16:03:16.303079 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.303053    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:03:16.376041 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.375950    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc1726b6-c780-4990-b864-a8be438c12f7-console-serving-cert\") pod \"cc1726b6-c780-4990-b864-a8be438c12f7\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") "
Jul 28 16:03:16.376041 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.376000    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc1726b6-c780-4990-b864-a8be438c12f7-console-oauth-config\") pod \"cc1726b6-c780-4990-b864-a8be438c12f7\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") "
Jul 28 16:03:16.376301 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.376083    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-console-config\") pod \"cc1726b6-c780-4990-b864-a8be438c12f7\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") "
Jul 28 16:03:16.376301 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.376136    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-service-ca\") pod \"cc1726b6-c780-4990-b864-a8be438c12f7\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") "
Jul 28 16:03:16.376301 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.376183    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcpb6\" (UniqueName: \"kubernetes.io/projected/cc1726b6-c780-4990-b864-a8be438c12f7-kube-api-access-gcpb6\") pod \"cc1726b6-c780-4990-b864-a8be438c12f7\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") "
Jul 28 16:03:16.376301 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.376227    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-oauth-serving-cert\") pod \"cc1726b6-c780-4990-b864-a8be438c12f7\" (UID: \"cc1726b6-c780-4990-b864-a8be438c12f7\") "
Jul 28 16:03:16.376518 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.376486    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-console-config" (OuterVolumeSpecName: "console-config") pod "cc1726b6-c780-4990-b864-a8be438c12f7" (UID: "cc1726b6-c780-4990-b864-a8be438c12f7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:03:16.376605 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.376578    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-service-ca" (OuterVolumeSpecName: "service-ca") pod "cc1726b6-c780-4990-b864-a8be438c12f7" (UID: "cc1726b6-c780-4990-b864-a8be438c12f7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:03:16.376657 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.376588    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cc1726b6-c780-4990-b864-a8be438c12f7" (UID: "cc1726b6-c780-4990-b864-a8be438c12f7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:03:16.378534 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.378509    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc1726b6-c780-4990-b864-a8be438c12f7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cc1726b6-c780-4990-b864-a8be438c12f7" (UID: "cc1726b6-c780-4990-b864-a8be438c12f7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:16.378762 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.378741    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc1726b6-c780-4990-b864-a8be438c12f7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cc1726b6-c780-4990-b864-a8be438c12f7" (UID: "cc1726b6-c780-4990-b864-a8be438c12f7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:16.378832 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.378769    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc1726b6-c780-4990-b864-a8be438c12f7-kube-api-access-gcpb6" (OuterVolumeSpecName: "kube-api-access-gcpb6") pod "cc1726b6-c780-4990-b864-a8be438c12f7" (UID: "cc1726b6-c780-4990-b864-a8be438c12f7"). InnerVolumeSpecName "kube-api-access-gcpb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:03:16.477790 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.477747    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-console-config\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:16.477790 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.477790    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-service-ca\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:16.478021 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.477804    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcpb6\" (UniqueName: \"kubernetes.io/projected/cc1726b6-c780-4990-b864-a8be438c12f7-kube-api-access-gcpb6\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:16.478021 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.477818    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc1726b6-c780-4990-b864-a8be438c12f7-oauth-serving-cert\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:16.478021 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.477832    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc1726b6-c780-4990-b864-a8be438c12f7-console-serving-cert\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:16.478021 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.477844    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc1726b6-c780-4990-b864-a8be438c12f7-console-oauth-config\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:16.997728 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.997695    2575 generic.go:356] "Generic (PLEG): container finished" podID="cc1726b6-c780-4990-b864-a8be438c12f7" containerID="248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36" exitCode=0
Jul 28 16:03:16.997909 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.997741    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675f96fd95-mk6xb"
Jul 28 16:03:16.997909 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.997782    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675f96fd95-mk6xb" event={"ID":"cc1726b6-c780-4990-b864-a8be438c12f7","Type":"ContainerDied","Data":"248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36"}
Jul 28 16:03:16.997909 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.997839    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675f96fd95-mk6xb" event={"ID":"cc1726b6-c780-4990-b864-a8be438c12f7","Type":"ContainerDied","Data":"1034842a4ed682fdf59b28b9426cb94bcc87202f26a3c952113ea9312a3b6edc"}
Jul 28 16:03:16.997909 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:16.997861    2575 scope.go:117] "RemoveContainer" containerID="248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36"
Jul 28 16:03:17.006348 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:17.006322    2575 scope.go:117] "RemoveContainer" containerID="248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36"
Jul 28 16:03:17.006693 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:03:17.006673    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36\": container with ID starting with 248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36 not found: ID does not exist" containerID="248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36"
Jul 28 16:03:17.006751 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:17.006710    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36"} err="failed to get container status \"248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36\": rpc error: code = NotFound desc = could not find container \"248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36\": container with ID starting with 248737dc67ca24bee1d1221640ddd30604a34708d5d7e26f292127ef7d740b36 not found: ID does not exist"
Jul 28 16:03:17.013290 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:17.013256    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-675f96fd95-mk6xb"]
Jul 28 16:03:17.017705 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:17.017673    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-675f96fd95-mk6xb"]
Jul 28 16:03:18.444057 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:18.444022    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc1726b6-c780-4990-b864-a8be438c12f7" path="/var/lib/kubelet/pods/cc1726b6-c780-4990-b864-a8be438c12f7/volumes"
Jul 28 16:03:29.825838 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:29.825786    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:29.845459 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:29.845430    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:30.063938 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:30.063904    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:43.676577 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:43.676533    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 16:03:43.677209 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:43.677122    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="alertmanager" containerID="cri-o://b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916" gracePeriod=120
Jul 28 16:03:43.677938 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:43.677576    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="prom-label-proxy" containerID="cri-o://fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29" gracePeriod=120
Jul 28 16:03:43.677938 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:43.677731    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="kube-rbac-proxy-metric" containerID="cri-o://20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541" gracePeriod=120
Jul 28 16:03:43.677938 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:43.677847    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="kube-rbac-proxy" containerID="cri-o://5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa" gracePeriod=120
Jul 28 16:03:43.678089 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:43.677956    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="config-reloader" containerID="cri-o://36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024" gracePeriod=120
Jul 28 16:03:43.678089 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:43.678010    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="kube-rbac-proxy-web" containerID="cri-o://2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791" gracePeriod=120
Jul 28 16:03:44.115036 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:44.115002    2575 generic.go:356] "Generic (PLEG): container finished" podID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerID="fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29" exitCode=0
Jul 28 16:03:44.115036 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:44.115028    2575 generic.go:356] "Generic (PLEG): container finished" podID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerID="20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541" exitCode=0
Jul 28 16:03:44.115036 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:44.115037    2575 generic.go:356] "Generic (PLEG): container finished" podID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerID="5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa" exitCode=0
Jul 28 16:03:44.115036 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:44.115044    2575 generic.go:356] "Generic (PLEG): container finished" podID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerID="36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024" exitCode=0
Jul 28 16:03:44.115036 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:44.115051    2575 generic.go:356] "Generic (PLEG): container finished" podID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerID="b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916" exitCode=0
Jul 28 16:03:44.115362 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:44.115079    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerDied","Data":"fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29"}
Jul 28 16:03:44.115362 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:44.115164    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerDied","Data":"20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541"}
Jul 28 16:03:44.115362 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:44.115191    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerDied","Data":"5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa"}
Jul 28 16:03:44.115362 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:44.115216    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerDied","Data":"36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024"}
Jul 28 16:03:44.115362 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:44.115241    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerDied","Data":"b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916"}
Jul 28 16:03:44.923887 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:44.923860    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:03:45.019882 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.019792    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-web-config\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.019882 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.019841    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-trusted-ca-bundle\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.019882 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.019875    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21031462-b95c-4f26-9e14-d8f115b0d6a7-tls-assets\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.020197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.019896    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-main-tls\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.020197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.019919    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-main-db\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.020197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.019946    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-cluster-tls-config\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.020197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.019968    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-config-volume\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.020197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.019984    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-metrics-client-ca\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.020197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.020009    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.020197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.020033    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcmkr\" (UniqueName: \"kubernetes.io/projected/21031462-b95c-4f26-9e14-d8f115b0d6a7-kube-api-access-vcmkr\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.020197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.020148    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.020707 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.020217    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy-web\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.020707 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.020257    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21031462-b95c-4f26-9e14-d8f115b0d6a7-config-out\") pod \"21031462-b95c-4f26-9e14-d8f115b0d6a7\" (UID: \"21031462-b95c-4f26-9e14-d8f115b0d6a7\") "
Jul 28 16:03:45.020707 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.020286    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 16:03:45.020707 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.020325    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:03:45.020707 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.020667    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.020707 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.020675    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:03:45.020707 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.020689    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21031462-b95c-4f26-9e14-d8f115b0d6a7-alertmanager-main-db\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.023167 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.023106    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21031462-b95c-4f26-9e14-d8f115b0d6a7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:03:45.023795 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.023762    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:45.024469 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.024357    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21031462-b95c-4f26-9e14-d8f115b0d6a7-kube-api-access-vcmkr" (OuterVolumeSpecName: "kube-api-access-vcmkr") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "kube-api-access-vcmkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:03:45.024469 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.024366    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-config-volume" (OuterVolumeSpecName: "config-volume") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:45.024469 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.024412    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21031462-b95c-4f26-9e14-d8f115b0d6a7-config-out" (OuterVolumeSpecName: "config-out") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 16:03:45.024640 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.024559    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:45.024697 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.024667    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:45.024750 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.024729    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:45.027625 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.027604    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:45.034393 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.034356    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-web-config" (OuterVolumeSpecName: "web-config") pod "21031462-b95c-4f26-9e14-d8f115b0d6a7" (UID: "21031462-b95c-4f26-9e14-d8f115b0d6a7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:45.121064 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121032    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.121064 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121054    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.121064 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121068    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21031462-b95c-4f26-9e14-d8f115b0d6a7-config-out\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.121365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121078    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-web-config\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.121365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121085    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21031462-b95c-4f26-9e14-d8f115b0d6a7-tls-assets\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.121365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121109    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-main-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.121365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121121    2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-cluster-tls-config\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.121365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121132    2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-config-volume\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.121365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121146    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21031462-b95c-4f26-9e14-d8f115b0d6a7-metrics-client-ca\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.121365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121160    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21031462-b95c-4f26-9e14-d8f115b0d6a7-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.121365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121171    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcmkr\" (UniqueName: \"kubernetes.io/projected/21031462-b95c-4f26-9e14-d8f115b0d6a7-kube-api-access-vcmkr\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:45.121673 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121648    2575 generic.go:356] "Generic (PLEG): container finished" podID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerID="2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791" exitCode=0
Jul 28 16:03:45.121745 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121723    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerDied","Data":"2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791"}
Jul 28 16:03:45.121782 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121747    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 16:03:45.121814 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121795    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21031462-b95c-4f26-9e14-d8f115b0d6a7","Type":"ContainerDied","Data":"13acec3635eae47455566be783c0017456fba3599c3a77174c6d223a1aee46b7"}
Jul 28 16:03:45.121847 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.121828    2575 scope.go:117] "RemoveContainer" containerID="fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29"
Jul 28 16:03:45.129844 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.129827    2575 scope.go:117] "RemoveContainer" containerID="20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541"
Jul 28 16:03:45.137283 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.137262    2575 scope.go:117] "RemoveContainer" containerID="5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa"
Jul 28 16:03:45.144266 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.144206    2575 scope.go:117] "RemoveContainer" containerID="2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791"
Jul 28 16:03:45.146341 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.146314    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 16:03:45.152243 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.152213    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 16:03:45.153409 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.153394    2575 scope.go:117] "RemoveContainer" containerID="36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024"
Jul 28 16:03:45.161133 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.161110    2575 scope.go:117] "RemoveContainer" containerID="b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916"
Jul 28 16:03:45.168739 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.168714    2575 scope.go:117] "RemoveContainer" containerID="c9fd8a65b1ceb73a5a201a15b43ba6f280dc1f3e48a7d8e1b1f798dbfe0d7031"
Jul 28 16:03:45.176452 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.176432    2575 scope.go:117] "RemoveContainer" containerID="fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29"
Jul 28 16:03:45.176759 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:03:45.176738    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29\": container with ID starting with fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29 not found: ID does not exist" containerID="fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29"
Jul 28 16:03:45.176843 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.176770    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29"} err="failed to get container status \"fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29\": rpc error: code = NotFound desc = could not find container \"fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29\": container with ID starting with fb1f83a1d5c2859f818367a0f0c05b1f3aa9850192aac97534483382d9722e29 not found: ID does not exist"
Jul 28 16:03:45.176843 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.176790    2575 scope.go:117] "RemoveContainer" containerID="20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541"
Jul 28 16:03:45.177006 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:03:45.176990    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541\": container with ID starting with 20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541 not found: ID does not exist" containerID="20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541"
Jul 28 16:03:45.177044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.177015    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541"} err="failed to get container status \"20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541\": rpc error: code = NotFound desc = could not find container \"20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541\": container with ID starting with 20bb28aaeb0996a0b7dcdb18f719de11359dfd7535f0000bf5a546fea95ee541 not found: ID does not exist"
Jul 28 16:03:45.177044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.177031    2575 scope.go:117] "RemoveContainer" containerID="5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa"
Jul 28 16:03:45.177288 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:03:45.177253    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa\": container with ID starting with 5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa not found: ID does not exist" containerID="5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa"
Jul 28 16:03:45.177333 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.177295    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa"} err="failed to get container status \"5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa\": rpc error: code = NotFound desc = could not find container \"5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa\": container with ID starting with 5128ab3443a6a2d10570948d0901c2c9b2668afb4b1fd792820f03f2f434bafa not found: ID does not exist"
Jul 28 16:03:45.177333 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.177310    2575 scope.go:117] "RemoveContainer" containerID="2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791"
Jul 28 16:03:45.177594 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:03:45.177569    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791\": container with ID starting with 2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791 not found: ID does not exist" containerID="2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791"
Jul 28 16:03:45.177665 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.177617    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791"} err="failed to get container status \"2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791\": rpc error: code = NotFound desc = could not find container \"2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791\": container with ID starting with 2d0fd004ef38dfa3800247f633d934c519d7d0d57f47996f8439b282702ee791 not found: ID does not exist"
Jul 28 16:03:45.177665 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.177640    2575 scope.go:117] "RemoveContainer" containerID="36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024"
Jul 28 16:03:45.177880 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:03:45.177862    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024\": container with ID starting with 36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024 not found: ID does not exist" containerID="36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024"
Jul 28 16:03:45.177918 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.177886    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024"} err="failed to get container status \"36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024\": rpc error: code = NotFound desc = could not find container \"36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024\": container with ID starting with 36755a4fe55a7688bb56f5128dd46603a22481a592155c19f4e59293a3efa024 not found: ID does not exist"
Jul 28 16:03:45.177918 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.177900    2575 scope.go:117] "RemoveContainer" containerID="b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916"
Jul 28 16:03:45.178149 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:03:45.178132    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916\": container with ID starting with b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916 not found: ID does not exist" containerID="b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916"
Jul 28 16:03:45.178201 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.178153    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916"} err="failed to get container status \"b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916\": rpc error: code = NotFound desc = could not find container \"b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916\": container with ID starting with b10545729cdc29bd8acacb5236c73a893a450b9918c502622215960d5422d916 not found: ID does not exist"
Jul 28 16:03:45.178201 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.178169    2575 scope.go:117] "RemoveContainer" containerID="c9fd8a65b1ceb73a5a201a15b43ba6f280dc1f3e48a7d8e1b1f798dbfe0d7031"
Jul 28 16:03:45.178418 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:03:45.178402    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9fd8a65b1ceb73a5a201a15b43ba6f280dc1f3e48a7d8e1b1f798dbfe0d7031\": container with ID starting with c9fd8a65b1ceb73a5a201a15b43ba6f280dc1f3e48a7d8e1b1f798dbfe0d7031 not found: ID does not exist" containerID="c9fd8a65b1ceb73a5a201a15b43ba6f280dc1f3e48a7d8e1b1f798dbfe0d7031"
Jul 28 16:03:45.178458 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:45.178421    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9fd8a65b1ceb73a5a201a15b43ba6f280dc1f3e48a7d8e1b1f798dbfe0d7031"} err="failed to get container status \"c9fd8a65b1ceb73a5a201a15b43ba6f280dc1f3e48a7d8e1b1f798dbfe0d7031\": rpc error: code = NotFound desc = could not find container \"c9fd8a65b1ceb73a5a201a15b43ba6f280dc1f3e48a7d8e1b1f798dbfe0d7031\": container with ID starting with c9fd8a65b1ceb73a5a201a15b43ba6f280dc1f3e48a7d8e1b1f798dbfe0d7031 not found: ID does not exist"
Jul 28 16:03:46.443294 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:46.443253    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" path="/var/lib/kubelet/pods/21031462-b95c-4f26-9e14-d8f115b0d6a7/volumes"
Jul 28 16:03:47.933708 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:47.933668    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 16:03:47.934163 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:47.934031    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="prometheus" containerID="cri-o://c5e2ebf3c76eb9806360e9a2c5ee78ae38b3aac688744680f205032dfcca99e9" gracePeriod=600
Jul 28 16:03:47.934163 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:47.934053    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="kube-rbac-proxy" containerID="cri-o://df6df453b7d30160c4bd00e2d7d30b017ef47b33c0214cde50b522a390909cf7" gracePeriod=600
Jul 28 16:03:47.934310 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:47.934165    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="thanos-sidecar" containerID="cri-o://cebf1efb08e6035cc41ea4e5db82a64584809a59e6239bf847b112d70eee4d94" gracePeriod=600
Jul 28 16:03:47.934310 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:47.934221    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="kube-rbac-proxy-thanos" containerID="cri-o://31cd7f92c64710d9049c1319f45ee7d670b94321679e5f688fd8d80c66f06895" gracePeriod=600
Jul 28 16:03:47.934310 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:47.934255    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="config-reloader" containerID="cri-o://89a961e00275ad769e1825eed8eb860c7598b3f9da2463ae8c6d1cf07167ffba" gracePeriod=600
Jul 28 16:03:47.934310 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:47.934290    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="kube-rbac-proxy-web" containerID="cri-o://f9b91832936784c3fa0272b995feb6a764cf20e44809d71a828b46609403d1c9" gracePeriod=600
Jul 28 16:03:48.143617 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.143582    2575 generic.go:356] "Generic (PLEG): container finished" podID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerID="31cd7f92c64710d9049c1319f45ee7d670b94321679e5f688fd8d80c66f06895" exitCode=0
Jul 28 16:03:48.143617 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.143611    2575 generic.go:356] "Generic (PLEG): container finished" podID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerID="df6df453b7d30160c4bd00e2d7d30b017ef47b33c0214cde50b522a390909cf7" exitCode=0
Jul 28 16:03:48.143617 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.143618    2575 generic.go:356] "Generic (PLEG): container finished" podID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerID="f9b91832936784c3fa0272b995feb6a764cf20e44809d71a828b46609403d1c9" exitCode=0
Jul 28 16:03:48.143617 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.143625    2575 generic.go:356] "Generic (PLEG): container finished" podID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerID="cebf1efb08e6035cc41ea4e5db82a64584809a59e6239bf847b112d70eee4d94" exitCode=0
Jul 28 16:03:48.143859 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.143631    2575 generic.go:356] "Generic (PLEG): container finished" podID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerID="89a961e00275ad769e1825eed8eb860c7598b3f9da2463ae8c6d1cf07167ffba" exitCode=0
Jul 28 16:03:48.143859 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.143636    2575 generic.go:356] "Generic (PLEG): container finished" podID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerID="c5e2ebf3c76eb9806360e9a2c5ee78ae38b3aac688744680f205032dfcca99e9" exitCode=0
Jul 28 16:03:48.143859 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.143656    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerDied","Data":"31cd7f92c64710d9049c1319f45ee7d670b94321679e5f688fd8d80c66f06895"}
Jul 28 16:03:48.143859 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.143726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerDied","Data":"df6df453b7d30160c4bd00e2d7d30b017ef47b33c0214cde50b522a390909cf7"}
Jul 28 16:03:48.143859 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.143752    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerDied","Data":"f9b91832936784c3fa0272b995feb6a764cf20e44809d71a828b46609403d1c9"}
Jul 28 16:03:48.143859 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.143779    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerDied","Data":"cebf1efb08e6035cc41ea4e5db82a64584809a59e6239bf847b112d70eee4d94"}
Jul 28 16:03:48.143859 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.143803    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerDied","Data":"89a961e00275ad769e1825eed8eb860c7598b3f9da2463ae8c6d1cf07167ffba"}
Jul 28 16:03:48.143859 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.143829    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerDied","Data":"c5e2ebf3c76eb9806360e9a2c5ee78ae38b3aac688744680f205032dfcca99e9"}
Jul 28 16:03:48.198139 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.198110    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:48.354442 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354401    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-web-config\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.354442 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354448    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-metrics-client-ca\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.354712 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354466    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p97rn\" (UniqueName: \"kubernetes.io/projected/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-kube-api-access-p97rn\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.354712 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354484    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-0\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.354712 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354503    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-trusted-ca-bundle\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.354712 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354607    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.354712 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354645    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-config\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.354712 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354673    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-grpc-tls\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.354712 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354699    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-thanos-prometheus-http-client-file\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.355049 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354733    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-db\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.355049 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354759    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.355049 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354801    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-kubelet-serving-ca-bundle\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.355049 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354836    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-kube-rbac-proxy\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.355049 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354868    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-1\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.355049 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354908    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-tls\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.355049 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354921    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:03:48.355049 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354933    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-metrics-client-certs\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.355049 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354963    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-2\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.355049 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354984    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:03:48.355049 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.354992    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-tls-assets\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.355049 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.355047    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-serving-certs-ca-bundle\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.355670 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.355077    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-config-out\") pod \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\" (UID: \"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9\") "
Jul 28 16:03:48.355731 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.355680    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:03:48.356035 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.356008    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:03:48.356214 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.356190    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.356281 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.356216    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-metrics-client-ca\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.356281 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.356232    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.356281 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.356245    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.357389 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.357063    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:03:48.357389 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.357087    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:03:48.357389 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.357375    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-config" (OuterVolumeSpecName: "config") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:48.357617 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.357374    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:03:48.357750 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.357674    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-kube-api-access-p97rn" (OuterVolumeSpecName: "kube-api-access-p97rn") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "kube-api-access-p97rn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:03:48.358409 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.358374    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:03:48.358625 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.358518    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-config-out" (OuterVolumeSpecName: "config-out") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 16:03:48.358625 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.358602    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 16:03:48.358865 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.358844    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:48.359431 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.359394    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:48.359587 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.359554    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:48.359723 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.359699    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:48.359785 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.359750    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:48.359872 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.359852    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:48.359904 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.359891    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:48.368410 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.368375    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-web-config" (OuterVolumeSpecName: "web-config") pod "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" (UID: "8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:03:48.457083 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457043    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457083 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457078    2575 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-kube-rbac-proxy\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457083 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457110    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457120    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457129    2575 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-metrics-client-certs\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457137    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457148    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-tls-assets\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457156    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-config-out\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457164    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-web-config\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457172    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p97rn\" (UniqueName: \"kubernetes.io/projected/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-kube-api-access-p97rn\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457180    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457189    2575 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-config\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457197    2575 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-grpc-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457205    2575 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457213    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-prometheus-k8s-db\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:48.457302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:48.457221    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:03:49.150886 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.150852    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9","Type":"ContainerDied","Data":"1c7c3c0ccae24c466aca517f86ba129064449c3f75ed79fbf5ff3572d235f4a6"}
Jul 28 16:03:49.151371 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.150930    2575 scope.go:117] "RemoveContainer" containerID="31cd7f92c64710d9049c1319f45ee7d670b94321679e5f688fd8d80c66f06895"
Jul 28 16:03:49.151371 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.150866    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.158727 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.158703    2575 scope.go:117] "RemoveContainer" containerID="df6df453b7d30160c4bd00e2d7d30b017ef47b33c0214cde50b522a390909cf7"
Jul 28 16:03:49.166025 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.166000    2575 scope.go:117] "RemoveContainer" containerID="f9b91832936784c3fa0272b995feb6a764cf20e44809d71a828b46609403d1c9"
Jul 28 16:03:49.172058 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.172031    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 16:03:49.174073 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.174053    2575 scope.go:117] "RemoveContainer" containerID="cebf1efb08e6035cc41ea4e5db82a64584809a59e6239bf847b112d70eee4d94"
Jul 28 16:03:49.179512 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.179486    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 16:03:49.182302 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.182284    2575 scope.go:117] "RemoveContainer" containerID="89a961e00275ad769e1825eed8eb860c7598b3f9da2463ae8c6d1cf07167ffba"
Jul 28 16:03:49.189871 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.189851    2575 scope.go:117] "RemoveContainer" containerID="c5e2ebf3c76eb9806360e9a2c5ee78ae38b3aac688744680f205032dfcca99e9"
Jul 28 16:03:49.198154 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.198134    2575 scope.go:117] "RemoveContainer" containerID="59b5ccbabdf82efbf6fee5ebfa89e947e09a2c91d549c1084a911eb483a970d0"
Jul 28 16:03:49.205434 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205403    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 16:03:49.205646 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205633    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="kube-rbac-proxy-thanos"
Jul 28 16:03:49.205692 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205647    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="kube-rbac-proxy-thanos"
Jul 28 16:03:49.205692 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205656    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="alertmanager"
Jul 28 16:03:49.205692 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205661    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="alertmanager"
Jul 28 16:03:49.205692 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205668    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="config-reloader"
Jul 28 16:03:49.205692 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205673    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="config-reloader"
Jul 28 16:03:49.205692 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205681    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="init-config-reloader"
Jul 28 16:03:49.205692 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205686    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="init-config-reloader"
Jul 28 16:03:49.205692 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205692    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="init-config-reloader"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205697    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="init-config-reloader"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205705    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="kube-rbac-proxy"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205710    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="kube-rbac-proxy"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205716    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc1726b6-c780-4990-b864-a8be438c12f7" containerName="console"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205721    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc1726b6-c780-4990-b864-a8be438c12f7" containerName="console"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205731    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="kube-rbac-proxy-web"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205738    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="kube-rbac-proxy-web"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205745    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="prometheus"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205750    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="prometheus"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205756    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="kube-rbac-proxy-web"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205761    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="kube-rbac-proxy-web"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205767    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="thanos-sidecar"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205772    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="thanos-sidecar"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205780    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="prom-label-proxy"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205785    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="prom-label-proxy"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205790    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="kube-rbac-proxy"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205795    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="kube-rbac-proxy"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205799    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="kube-rbac-proxy-metric"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205804    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="kube-rbac-proxy-metric"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205810    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="config-reloader"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205815    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="config-reloader"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205852    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="config-reloader"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205859    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="kube-rbac-proxy-metric"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205865    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="alertmanager"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205870    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="prometheus"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205875    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="kube-rbac-proxy-thanos"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205880    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="kube-rbac-proxy-web"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205885    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="kube-rbac-proxy"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205889    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="config-reloader"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205894    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc1726b6-c780-4990-b864-a8be438c12f7" containerName="console"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205899    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="21031462-b95c-4f26-9e14-d8f115b0d6a7" containerName="prom-label-proxy"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205904    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="kube-rbac-proxy-web"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205910    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="kube-rbac-proxy"
Jul 28 16:03:49.205939 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.205915    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" containerName="thanos-sidecar"
Jul 28 16:03:49.211262 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.210822    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.213568 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.213545    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 16:03:49.214160 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.214138    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5iuqekapmmjgl\""
Jul 28 16:03:49.214242 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.214182    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 16:03:49.214362 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.214345    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 16:03:49.214420 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.214359    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 16:03:49.214476 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.214465    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 16:03:49.214604 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.214588    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 16:03:49.214671 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.214604    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 16:03:49.214671 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.214641    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 16:03:49.214776 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.214642    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 16:03:49.214776 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.214719    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 16:03:49.214860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.214772    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 16:03:49.214860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.214799    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2txww\""
Jul 28 16:03:49.215325 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.215307    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 16:03:49.217376 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.217355    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 16:03:49.220813 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.220757    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 16:03:49.230285 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.230252    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 16:03:49.261339 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261302    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261339 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261339    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261611 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261361    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261611 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261379    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261611 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261425    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cae1800-bf89-4c00-a732-4a9a6b4de481-config-out\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261611 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261472    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cae1800-bf89-4c00-a732-4a9a6b4de481-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261611 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261502    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261611 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261528    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261611 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261551    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261611 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261576    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-web-config\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261936 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261632    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261936 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261664    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261936 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261697    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261936 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261723    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261936 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261756    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261936 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261814    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261936 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261841    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261936 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261863    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r446w\" (UniqueName: \"kubernetes.io/projected/6cae1800-bf89-4c00-a732-4a9a6b4de481-kube-api-access-r446w\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261936 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261885    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.261936 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.261902    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-config\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.363184 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363146    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.363184 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363192    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.363452 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363329    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.363452 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363366    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r446w\" (UniqueName: \"kubernetes.io/projected/6cae1800-bf89-4c00-a732-4a9a6b4de481-kube-api-access-r446w\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.363452 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363389    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.363452 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363416    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-config\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.363452 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363440    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363456    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363488    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363519    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363550    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cae1800-bf89-4c00-a732-4a9a6b4de481-config-out\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363587    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cae1800-bf89-4c00-a732-4a9a6b4de481-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363618    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363647    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363693    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-web-config\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363727    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363771    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363791    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363812    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363845    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.363850    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.364406 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.364143    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.365208 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.364834    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.366944 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.365278    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.366944 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.365358    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.366944 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.366160    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.368036 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.367139    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.368036 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.367178    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.368036 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.367736    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6cae1800-bf89-4c00-a732-4a9a6b4de481-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.368036 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.367816    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cae1800-bf89-4c00-a732-4a9a6b4de481-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.368633 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.368601    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cae1800-bf89-4c00-a732-4a9a6b4de481-config-out\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.368810 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.368792    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.368874 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.368805    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.368919 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.368901    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-web-config\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.369026 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.369007    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-config\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.369341 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.369319    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.369495 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.369474    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.370248 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.370228    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6cae1800-bf89-4c00-a732-4a9a6b4de481-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.371948 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.371925    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r446w\" (UniqueName: \"kubernetes.io/projected/6cae1800-bf89-4c00-a732-4a9a6b4de481-kube-api-access-r446w\") pod \"prometheus-k8s-0\" (UID: \"6cae1800-bf89-4c00-a732-4a9a6b4de481\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.522165 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.522066    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:03:49.665488 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:49.665454    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 16:03:49.669143 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:03:49.669113    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cae1800_bf89_4c00_a732_4a9a6b4de481.slice/crio-2c4568cc45e1960911a77604fe89728d56547ebb77a0864e8f6650e4657548e6 WatchSource:0}: Error finding container 2c4568cc45e1960911a77604fe89728d56547ebb77a0864e8f6650e4657548e6: Status 404 returned error can't find the container with id 2c4568cc45e1960911a77604fe89728d56547ebb77a0864e8f6650e4657548e6
Jul 28 16:03:50.156397 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:50.156362    2575 generic.go:356] "Generic (PLEG): container finished" podID="6cae1800-bf89-4c00-a732-4a9a6b4de481" containerID="a2635dba857cfba0afe0013f1b3d8baa424c46ef0b1ecce4fa7eea34bddf6e9a" exitCode=0
Jul 28 16:03:50.156793 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:50.156418    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6cae1800-bf89-4c00-a732-4a9a6b4de481","Type":"ContainerDied","Data":"a2635dba857cfba0afe0013f1b3d8baa424c46ef0b1ecce4fa7eea34bddf6e9a"}
Jul 28 16:03:50.156793 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:50.156468    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6cae1800-bf89-4c00-a732-4a9a6b4de481","Type":"ContainerStarted","Data":"2c4568cc45e1960911a77604fe89728d56547ebb77a0864e8f6650e4657548e6"}
Jul 28 16:03:50.445744 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:50.445705    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9" path="/var/lib/kubelet/pods/8e3e8fcd-e8dc-42e1-a23a-3fe6ae7b2fc9/volumes"
Jul 28 16:03:51.164576 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:51.164535    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6cae1800-bf89-4c00-a732-4a9a6b4de481","Type":"ContainerStarted","Data":"62ca122f0be54d497fdf7aaa45fdb51df15508c70bf33e853f74a4e7ab6238b7"}
Jul 28 16:03:51.164974 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:51.164630    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6cae1800-bf89-4c00-a732-4a9a6b4de481","Type":"ContainerStarted","Data":"b7b33a744010d7af63c625673150105e3e2a526daa5c0529549c4ecbfc0dd8f1"}
Jul 28 16:03:51.164974 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:51.164667    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6cae1800-bf89-4c00-a732-4a9a6b4de481","Type":"ContainerStarted","Data":"14d4f8f87ae3f75293d379a0ba37db3e538a0346d16b704dfbb177b4a59f7e07"}
Jul 28 16:03:51.164974 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:51.164698    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6cae1800-bf89-4c00-a732-4a9a6b4de481","Type":"ContainerStarted","Data":"13e90c73e24f694c9d8c5d35706432e890cc255cce31ef398bb3805b8dc9ec6d"}
Jul 28 16:03:51.164974 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:51.164721    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6cae1800-bf89-4c00-a732-4a9a6b4de481","Type":"ContainerStarted","Data":"13179f249d4f56974ecb5a12cbb4bbb98e952f3d6eb64b09a933db953f228b70"}
Jul 28 16:03:51.164974 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:51.164744    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6cae1800-bf89-4c00-a732-4a9a6b4de481","Type":"ContainerStarted","Data":"b3245a6917446a8db7130a35568bb01fe9c46e24f9b6dd6ae0a3768630c906e5"}
Jul 28 16:03:51.195057 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:51.194988    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.173735939 podStartE2EDuration="2.194964124s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.228185ms" isStatefulPod=false podCreationTimestamp="2026-07-28 16:03:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:03:51.192540758 +0000 UTC m=+157.336919312" watchObservedRunningTime="2026-07-28 16:03:51.194964124 +0000 UTC m=+157.339342676"
Jul 28 16:03:54.522744 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:03:54.522688    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:04:49.523083 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:04:49.523044    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:04:49.539047 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:04:49.539015    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:04:50.399944 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:04:50.399912    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 16:06:14.264862 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:06:14.262986    2575 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 16:07:00.124923 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.124885    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-8j76m"]
Jul 28 16:07:00.128460 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.128443    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-654c557856-8j76m"
Jul 28 16:07:00.130530 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.130504    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 28 16:07:00.130677 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.130533    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 28 16:07:00.130677 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.130606    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-lws2w\""
Jul 28 16:07:00.130916 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.130903    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 28 16:07:00.138069 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.138043    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-8j76m"]
Jul 28 16:07:00.188324 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.188289    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2c555b7-83da-41ef-a8c7-358ac31a9dff-cert\") pod \"llmisvc-controller-manager-654c557856-8j76m\" (UID: \"a2c555b7-83da-41ef-a8c7-358ac31a9dff\") " pod="kserve/llmisvc-controller-manager-654c557856-8j76m"
Jul 28 16:07:00.188499 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.188345    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdrtc\" (UniqueName: \"kubernetes.io/projected/a2c555b7-83da-41ef-a8c7-358ac31a9dff-kube-api-access-jdrtc\") pod \"llmisvc-controller-manager-654c557856-8j76m\" (UID: \"a2c555b7-83da-41ef-a8c7-358ac31a9dff\") " pod="kserve/llmisvc-controller-manager-654c557856-8j76m"
Jul 28 16:07:00.288984 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.288947    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2c555b7-83da-41ef-a8c7-358ac31a9dff-cert\") pod \"llmisvc-controller-manager-654c557856-8j76m\" (UID: \"a2c555b7-83da-41ef-a8c7-358ac31a9dff\") " pod="kserve/llmisvc-controller-manager-654c557856-8j76m"
Jul 28 16:07:00.289195 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.289007    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdrtc\" (UniqueName: \"kubernetes.io/projected/a2c555b7-83da-41ef-a8c7-358ac31a9dff-kube-api-access-jdrtc\") pod \"llmisvc-controller-manager-654c557856-8j76m\" (UID: \"a2c555b7-83da-41ef-a8c7-358ac31a9dff\") " pod="kserve/llmisvc-controller-manager-654c557856-8j76m"
Jul 28 16:07:00.291578 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.291539    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2c555b7-83da-41ef-a8c7-358ac31a9dff-cert\") pod \"llmisvc-controller-manager-654c557856-8j76m\" (UID: \"a2c555b7-83da-41ef-a8c7-358ac31a9dff\") " pod="kserve/llmisvc-controller-manager-654c557856-8j76m"
Jul 28 16:07:00.298450 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.298424    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdrtc\" (UniqueName: \"kubernetes.io/projected/a2c555b7-83da-41ef-a8c7-358ac31a9dff-kube-api-access-jdrtc\") pod \"llmisvc-controller-manager-654c557856-8j76m\" (UID: \"a2c555b7-83da-41ef-a8c7-358ac31a9dff\") " pod="kserve/llmisvc-controller-manager-654c557856-8j76m"
Jul 28 16:07:00.439204 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.439079    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-654c557856-8j76m"
Jul 28 16:07:00.569987 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.569954    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-8j76m"]
Jul 28 16:07:00.573458 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:07:00.573425    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda2c555b7_83da_41ef_a8c7_358ac31a9dff.slice/crio-15b07a7787224b902b95eb932e1a20d714e7dbbe8bdf1a7afd2cd13e7eddd618 WatchSource:0}: Error finding container 15b07a7787224b902b95eb932e1a20d714e7dbbe8bdf1a7afd2cd13e7eddd618: Status 404 returned error can't find the container with id 15b07a7787224b902b95eb932e1a20d714e7dbbe8bdf1a7afd2cd13e7eddd618
Jul 28 16:07:00.575040 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.575019    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 16:07:00.829201 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:00.829167    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-8j76m" event={"ID":"a2c555b7-83da-41ef-a8c7-358ac31a9dff","Type":"ContainerStarted","Data":"15b07a7787224b902b95eb932e1a20d714e7dbbe8bdf1a7afd2cd13e7eddd618"}
Jul 28 16:07:05.848665 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:05.848623    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-8j76m" event={"ID":"a2c555b7-83da-41ef-a8c7-358ac31a9dff","Type":"ContainerStarted","Data":"735bb65e7e8d44d5a433086b6edd1b5b0565b148127ab1ab3444f15db6ce5927"}
Jul 28 16:07:05.849122 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:05.848765    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-654c557856-8j76m"
Jul 28 16:07:05.863954 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:05.863900    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-654c557856-8j76m" podStartSLOduration=1.550531272 podStartE2EDuration="5.863882857s" totalImagesPullingTime="4.313351585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:07:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:07:05.86220302 +0000 UTC m=+352.006581585" watchObservedRunningTime="2026-07-28 16:07:05.863882857 +0000 UTC m=+352.008261410"
Jul 28 16:07:36.854585 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:07:36.854552    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-654c557856-8j76m"
Jul 28 16:08:35.311040 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.310950    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-kdj2d"]
Jul 28 16:08:35.313586 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.313568    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-kdj2d"
Jul 28 16:08:35.315683 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.315655    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 28 16:08:35.315862 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.315845    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-lswf4\""
Jul 28 16:08:35.322511 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.322475    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-kdj2d"]
Jul 28 16:08:35.363641 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.363604    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/717d4ef7-07af-496b-a727-dbdb172625de-model-cache\") pod \"s3-init-kdj2d\" (UID: \"717d4ef7-07af-496b-a727-dbdb172625de\") " pod="kserve/s3-init-kdj2d"
Jul 28 16:08:35.363831 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.363661    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4vnb\" (UniqueName: \"kubernetes.io/projected/717d4ef7-07af-496b-a727-dbdb172625de-kube-api-access-w4vnb\") pod \"s3-init-kdj2d\" (UID: \"717d4ef7-07af-496b-a727-dbdb172625de\") " pod="kserve/s3-init-kdj2d"
Jul 28 16:08:35.464399 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.464358    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vnb\" (UniqueName: \"kubernetes.io/projected/717d4ef7-07af-496b-a727-dbdb172625de-kube-api-access-w4vnb\") pod \"s3-init-kdj2d\" (UID: \"717d4ef7-07af-496b-a727-dbdb172625de\") " pod="kserve/s3-init-kdj2d"
Jul 28 16:08:35.464609 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.464427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/717d4ef7-07af-496b-a727-dbdb172625de-model-cache\") pod \"s3-init-kdj2d\" (UID: \"717d4ef7-07af-496b-a727-dbdb172625de\") " pod="kserve/s3-init-kdj2d"
Jul 28 16:08:35.464786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.464769    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/717d4ef7-07af-496b-a727-dbdb172625de-model-cache\") pod \"s3-init-kdj2d\" (UID: \"717d4ef7-07af-496b-a727-dbdb172625de\") " pod="kserve/s3-init-kdj2d"
Jul 28 16:08:35.472394 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.472362    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4vnb\" (UniqueName: \"kubernetes.io/projected/717d4ef7-07af-496b-a727-dbdb172625de-kube-api-access-w4vnb\") pod \"s3-init-kdj2d\" (UID: \"717d4ef7-07af-496b-a727-dbdb172625de\") " pod="kserve/s3-init-kdj2d"
Jul 28 16:08:35.631203 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.631081    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-kdj2d"
Jul 28 16:08:35.765213 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:35.765182    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-kdj2d"]
Jul 28 16:08:35.769448 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:08:35.769416    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod717d4ef7_07af_496b_a727_dbdb172625de.slice/crio-67ace11ddf0c4255d0ceae769dd9eec3a0266d40d111f858c349fc38bdae9e39 WatchSource:0}: Error finding container 67ace11ddf0c4255d0ceae769dd9eec3a0266d40d111f858c349fc38bdae9e39: Status 404 returned error can't find the container with id 67ace11ddf0c4255d0ceae769dd9eec3a0266d40d111f858c349fc38bdae9e39
Jul 28 16:08:36.168756 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:36.168717    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-kdj2d" event={"ID":"717d4ef7-07af-496b-a727-dbdb172625de","Type":"ContainerStarted","Data":"67ace11ddf0c4255d0ceae769dd9eec3a0266d40d111f858c349fc38bdae9e39"}
Jul 28 16:08:40.185494 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:40.185388    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-kdj2d" event={"ID":"717d4ef7-07af-496b-a727-dbdb172625de","Type":"ContainerStarted","Data":"2f4154d0e699dce9d1a28950060eaae06253adccdba75971f93d59bcabc733e5"}
Jul 28 16:08:42.195196 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:42.195086    2575 generic.go:356] "Generic (PLEG): container finished" podID="717d4ef7-07af-496b-a727-dbdb172625de" containerID="2f4154d0e699dce9d1a28950060eaae06253adccdba75971f93d59bcabc733e5" exitCode=0
Jul 28 16:08:42.195196 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:42.195128    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-kdj2d" event={"ID":"717d4ef7-07af-496b-a727-dbdb172625de","Type":"ContainerDied","Data":"2f4154d0e699dce9d1a28950060eaae06253adccdba75971f93d59bcabc733e5"}
Jul 28 16:08:47.220658 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:47.220619    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-kdj2d" event={"ID":"717d4ef7-07af-496b-a727-dbdb172625de","Type":"ContainerStarted","Data":"70ff834c964c9eec32755bd0b0417e832539aeb9da1f9a1a6872255d44b188d6"}
Jul 28 16:08:47.238962 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:08:47.238901    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-kdj2d" podStartSLOduration=1.723382656 podStartE2EDuration="12.238883712s" totalImagesPullingTime="8.626579261s" totalInitContainerRuntime="1.888921795s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:08:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:08:47.237914384 +0000 UTC m=+453.382292938" watchObservedRunningTime="2026-07-28 16:08:47.238883712 +0000 UTC m=+453.383262266"
Jul 28 16:09:52.449849 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:09:52.449762    2575 generic.go:356] "Generic (PLEG): container finished" podID="717d4ef7-07af-496b-a727-dbdb172625de" containerID="70ff834c964c9eec32755bd0b0417e832539aeb9da1f9a1a6872255d44b188d6" exitCode=0
Jul 28 16:09:52.449849 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:09:52.449829    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-kdj2d" event={"ID":"717d4ef7-07af-496b-a727-dbdb172625de","Type":"ContainerDied","Data":"70ff834c964c9eec32755bd0b0417e832539aeb9da1f9a1a6872255d44b188d6"}
Jul 28 16:09:53.578951 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:09:53.578925    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-kdj2d"
Jul 28 16:09:53.710448 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:09:53.710358    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4vnb\" (UniqueName: \"kubernetes.io/projected/717d4ef7-07af-496b-a727-dbdb172625de-kube-api-access-w4vnb\") pod \"717d4ef7-07af-496b-a727-dbdb172625de\" (UID: \"717d4ef7-07af-496b-a727-dbdb172625de\") "
Jul 28 16:09:53.710448 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:09:53.710435    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/717d4ef7-07af-496b-a727-dbdb172625de-model-cache\") pod \"717d4ef7-07af-496b-a727-dbdb172625de\" (UID: \"717d4ef7-07af-496b-a727-dbdb172625de\") "
Jul 28 16:09:53.712612 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:09:53.712583    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/717d4ef7-07af-496b-a727-dbdb172625de-kube-api-access-w4vnb" (OuterVolumeSpecName: "kube-api-access-w4vnb") pod "717d4ef7-07af-496b-a727-dbdb172625de" (UID: "717d4ef7-07af-496b-a727-dbdb172625de"). InnerVolumeSpecName "kube-api-access-w4vnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:09:53.729696 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:09:53.729653    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/717d4ef7-07af-496b-a727-dbdb172625de-model-cache" (OuterVolumeSpecName: "model-cache") pod "717d4ef7-07af-496b-a727-dbdb172625de" (UID: "717d4ef7-07af-496b-a727-dbdb172625de"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 16:09:53.810933 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:09:53.810898    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/717d4ef7-07af-496b-a727-dbdb172625de-model-cache\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:09:53.810933 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:09:53.810930    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4vnb\" (UniqueName: \"kubernetes.io/projected/717d4ef7-07af-496b-a727-dbdb172625de-kube-api-access-w4vnb\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:09:54.458333 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:09:54.458304    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-kdj2d"
Jul 28 16:09:54.458508 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:09:54.458296    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-kdj2d" event={"ID":"717d4ef7-07af-496b-a727-dbdb172625de","Type":"ContainerDied","Data":"67ace11ddf0c4255d0ceae769dd9eec3a0266d40d111f858c349fc38bdae9e39"}
Jul 28 16:09:54.458508 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:09:54.458430    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67ace11ddf0c4255d0ceae769dd9eec3a0266d40d111f858c349fc38bdae9e39"
Jul 28 16:10:07.190952 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.190864    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"]
Jul 28 16:10:07.191423 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.191166    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="717d4ef7-07af-496b-a727-dbdb172625de" containerName="download-hf-model"
Jul 28 16:10:07.191423 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.191178    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="717d4ef7-07af-496b-a727-dbdb172625de" containerName="download-hf-model"
Jul 28 16:10:07.191423 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.191189    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="717d4ef7-07af-496b-a727-dbdb172625de" containerName="s3-init"
Jul 28 16:10:07.191423 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.191195    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="717d4ef7-07af-496b-a727-dbdb172625de" containerName="s3-init"
Jul 28 16:10:07.191423 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.191252    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="717d4ef7-07af-496b-a727-dbdb172625de" containerName="s3-init"
Jul 28 16:10:07.194287 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.194265    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.197003 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.196970    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\""
Jul 28 16:10:07.197152 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.197041    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-k6dfd\""
Jul 28 16:10:07.197457 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.197439    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\""
Jul 28 16:10:07.197529 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.197468    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 28 16:10:07.197723 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.197707    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 28 16:10:07.197793 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.197757    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 28 16:10:07.198028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.198012    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 28 16:10:07.211571 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.211539    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"]
Jul 28 16:10:07.215009 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.214973    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f237da2-64e5-4047-99e1-56bcce0f6e24-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.215186 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.215028    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdjfm\" (UniqueName: \"kubernetes.io/projected/3f237da2-64e5-4047-99e1-56bcce0f6e24-kube-api-access-cdjfm\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.215186 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.215067    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f237da2-64e5-4047-99e1-56bcce0f6e24-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.215186 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.215128    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f237da2-64e5-4047-99e1-56bcce0f6e24-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.215186 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.215155    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f237da2-64e5-4047-99e1-56bcce0f6e24-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.297970 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.297939    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"]
Jul 28 16:10:07.301202 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.301187    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.303231 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.303201    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\""
Jul 28 16:10:07.303231 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.303223    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\""
Jul 28 16:10:07.310937 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.310914    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"]
Jul 28 16:10:07.315747 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.315721    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/89d15a04-9b9b-45c3-9dee-188d150229b7-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.315869 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.315765    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/89d15a04-9b9b-45c3-9dee-188d150229b7-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.315869 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.315802    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/89d15a04-9b9b-45c3-9dee-188d150229b7-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.315869 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.315855    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f237da2-64e5-4047-99e1-56bcce0f6e24-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.316014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.315883    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdjfm\" (UniqueName: \"kubernetes.io/projected/3f237da2-64e5-4047-99e1-56bcce0f6e24-kube-api-access-cdjfm\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.316014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.315911    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/89d15a04-9b9b-45c3-9dee-188d150229b7-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.316014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.315947    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f237da2-64e5-4047-99e1-56bcce0f6e24-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.316014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.315980    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blk6k\" (UniqueName: \"kubernetes.io/projected/89d15a04-9b9b-45c3-9dee-188d150229b7-kube-api-access-blk6k\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.316230 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.316087    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f237da2-64e5-4047-99e1-56bcce0f6e24-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.316230 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.316160    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f237da2-64e5-4047-99e1-56bcce0f6e24-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.316468 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.316436    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f237da2-64e5-4047-99e1-56bcce0f6e24-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.316629 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.316610    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f237da2-64e5-4047-99e1-56bcce0f6e24-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.316738 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.316720    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f237da2-64e5-4047-99e1-56bcce0f6e24-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.318477 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.318455    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f237da2-64e5-4047-99e1-56bcce0f6e24-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.325044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.325022    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdjfm\" (UniqueName: \"kubernetes.io/projected/3f237da2-64e5-4047-99e1-56bcce0f6e24-kube-api-access-cdjfm\") pod \"isvc-sklearn-graph-1-predictor-84c58bd689-44kr7\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.416784 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.416732    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/89d15a04-9b9b-45c3-9dee-188d150229b7-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.416784 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.416783    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/89d15a04-9b9b-45c3-9dee-188d150229b7-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.417047 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.416831    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/89d15a04-9b9b-45c3-9dee-188d150229b7-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.417047 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.416880    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/89d15a04-9b9b-45c3-9dee-188d150229b7-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.417047 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:10:07.416894    2575 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-xgboost-graph-predictor-serving-cert: secret "isvc-xgboost-graph-predictor-serving-cert" not found
Jul 28 16:10:07.417047 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.416915    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blk6k\" (UniqueName: \"kubernetes.io/projected/89d15a04-9b9b-45c3-9dee-188d150229b7-kube-api-access-blk6k\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.417047 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:10:07.416974    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89d15a04-9b9b-45c3-9dee-188d150229b7-proxy-tls podName:89d15a04-9b9b-45c3-9dee-188d150229b7 nodeName:}" failed. No retries permitted until 2026-07-28 16:10:07.916952563 +0000 UTC m=+534.061331107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/89d15a04-9b9b-45c3-9dee-188d150229b7-proxy-tls") pod "isvc-xgboost-graph-predictor-555fc69d56-6xnd2" (UID: "89d15a04-9b9b-45c3-9dee-188d150229b7") : secret "isvc-xgboost-graph-predictor-serving-cert" not found
Jul 28 16:10:07.417404 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.417276    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/89d15a04-9b9b-45c3-9dee-188d150229b7-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.417851 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.417828    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/89d15a04-9b9b-45c3-9dee-188d150229b7-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.417936 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.417913    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/89d15a04-9b9b-45c3-9dee-188d150229b7-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.425640 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.425616    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blk6k\" (UniqueName: \"kubernetes.io/projected/89d15a04-9b9b-45c3-9dee-188d150229b7-kube-api-access-blk6k\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.504967 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.504931    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:07.571713 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.571598    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"]
Jul 28 16:10:07.575251 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.575230    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.577290 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.577269    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\""
Jul 28 16:10:07.577424 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.577402    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\""
Jul 28 16:10:07.585846 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.585821    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"]
Jul 28 16:10:07.618979 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.618949    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a85b88d2-526f-4050-a7a4-84425d6173fd-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.619182 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.619007    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a85b88d2-526f-4050-a7a4-84425d6173fd-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.619182 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.619041    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8xd4\" (UniqueName: \"kubernetes.io/projected/a85b88d2-526f-4050-a7a4-84425d6173fd-kube-api-access-f8xd4\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.619182 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.619130    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a85b88d2-526f-4050-a7a4-84425d6173fd-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.619312 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.619228    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a85b88d2-526f-4050-a7a4-84425d6173fd-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.652053 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.652022    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"]
Jul 28 16:10:07.654544 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:10:07.654520    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f237da2_64e5_4047_99e1_56bcce0f6e24.slice/crio-86e6a391d8430e9929b24a4c5c4f1d060ca077404f2f3b4b8c13036dfce13287 WatchSource:0}: Error finding container 86e6a391d8430e9929b24a4c5c4f1d060ca077404f2f3b4b8c13036dfce13287: Status 404 returned error can't find the container with id 86e6a391d8430e9929b24a4c5c4f1d060ca077404f2f3b4b8c13036dfce13287
Jul 28 16:10:07.720501 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.720469    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a85b88d2-526f-4050-a7a4-84425d6173fd-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.720676 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.720521    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a85b88d2-526f-4050-a7a4-84425d6173fd-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.720676 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.720579    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a85b88d2-526f-4050-a7a4-84425d6173fd-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.720676 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.720605    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8xd4\" (UniqueName: \"kubernetes.io/projected/a85b88d2-526f-4050-a7a4-84425d6173fd-kube-api-access-f8xd4\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.720676 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.720637    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a85b88d2-526f-4050-a7a4-84425d6173fd-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.720916 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:10:07.720693    2575 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-serving-cert: secret "isvc-sklearn-graph-2-predictor-serving-cert" not found
Jul 28 16:10:07.720916 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:10:07.720759    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a85b88d2-526f-4050-a7a4-84425d6173fd-proxy-tls podName:a85b88d2-526f-4050-a7a4-84425d6173fd nodeName:}" failed. No retries permitted until 2026-07-28 16:10:08.220740446 +0000 UTC m=+534.365118978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/a85b88d2-526f-4050-a7a4-84425d6173fd-proxy-tls") pod "isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" (UID: "a85b88d2-526f-4050-a7a4-84425d6173fd") : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found
Jul 28 16:10:07.721081 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.720928    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a85b88d2-526f-4050-a7a4-84425d6173fd-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.721292 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.721274    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a85b88d2-526f-4050-a7a4-84425d6173fd-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.721373 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.721332    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a85b88d2-526f-4050-a7a4-84425d6173fd-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.728769 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.728738    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8xd4\" (UniqueName: \"kubernetes.io/projected/a85b88d2-526f-4050-a7a4-84425d6173fd-kube-api-access-f8xd4\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:07.923245 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.923142    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/89d15a04-9b9b-45c3-9dee-188d150229b7-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:07.925474 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:07.925447    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/89d15a04-9b9b-45c3-9dee-188d150229b7-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6xnd2\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:08.211858 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:08.211755    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:08.224943 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:08.224906    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a85b88d2-526f-4050-a7a4-84425d6173fd-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:08.227527 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:08.227501    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a85b88d2-526f-4050-a7a4-84425d6173fd-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:08.350265 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:08.350071    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"]
Jul 28 16:10:08.352919 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:10:08.352877    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89d15a04_9b9b_45c3_9dee_188d150229b7.slice/crio-2e954cb92bb702bb63f9f9a2b78d98d018d61eec9d3f9cfc7205d6e35091cc88 WatchSource:0}: Error finding container 2e954cb92bb702bb63f9f9a2b78d98d018d61eec9d3f9cfc7205d6e35091cc88: Status 404 returned error can't find the container with id 2e954cb92bb702bb63f9f9a2b78d98d018d61eec9d3f9cfc7205d6e35091cc88
Jul 28 16:10:08.486214 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:08.486180    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:08.508023 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:08.507980    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" event={"ID":"89d15a04-9b9b-45c3-9dee-188d150229b7","Type":"ContainerStarted","Data":"0f386a5704487e795c2c27e8ce2a8b1d7f0f367fc3d9a1c5708670f84a54520c"}
Jul 28 16:10:08.508239 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:08.508050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" event={"ID":"89d15a04-9b9b-45c3-9dee-188d150229b7","Type":"ContainerStarted","Data":"2e954cb92bb702bb63f9f9a2b78d98d018d61eec9d3f9cfc7205d6e35091cc88"}
Jul 28 16:10:08.509550 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:08.509525    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" event={"ID":"3f237da2-64e5-4047-99e1-56bcce0f6e24","Type":"ContainerStarted","Data":"99b3c592a4671325805ddfc64683c30d5b564f90aa4b977a9ff416635e9bf230"}
Jul 28 16:10:08.509659 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:08.509573    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" event={"ID":"3f237da2-64e5-4047-99e1-56bcce0f6e24","Type":"ContainerStarted","Data":"86e6a391d8430e9929b24a4c5c4f1d060ca077404f2f3b4b8c13036dfce13287"}
Jul 28 16:10:08.630372 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:08.630069    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"]
Jul 28 16:10:08.633024 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:10:08.632986    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda85b88d2_526f_4050_a7a4_84425d6173fd.slice/crio-19177cbf4d21aeb11049b454c511be672003f6153c2987e8b4917c68acb1a266 WatchSource:0}: Error finding container 19177cbf4d21aeb11049b454c511be672003f6153c2987e8b4917c68acb1a266: Status 404 returned error can't find the container with id 19177cbf4d21aeb11049b454c511be672003f6153c2987e8b4917c68acb1a266
Jul 28 16:10:09.515016 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:09.514972    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" event={"ID":"a85b88d2-526f-4050-a7a4-84425d6173fd","Type":"ContainerStarted","Data":"c855a2db6ae3fc9d913bcd29629574885d42accc0b7eb115599e5e5ec35809c7"}
Jul 28 16:10:09.515536 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:09.515036    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" event={"ID":"a85b88d2-526f-4050-a7a4-84425d6173fd","Type":"ContainerStarted","Data":"19177cbf4d21aeb11049b454c511be672003f6153c2987e8b4917c68acb1a266"}
Jul 28 16:10:12.528448 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:12.528353    2575 generic.go:356] "Generic (PLEG): container finished" podID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerID="c855a2db6ae3fc9d913bcd29629574885d42accc0b7eb115599e5e5ec35809c7" exitCode=0
Jul 28 16:10:12.528448 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:12.528421    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" event={"ID":"a85b88d2-526f-4050-a7a4-84425d6173fd","Type":"ContainerDied","Data":"c855a2db6ae3fc9d913bcd29629574885d42accc0b7eb115599e5e5ec35809c7"}
Jul 28 16:10:12.530163 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:12.530039    2575 generic.go:356] "Generic (PLEG): container finished" podID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerID="0f386a5704487e795c2c27e8ce2a8b1d7f0f367fc3d9a1c5708670f84a54520c" exitCode=0
Jul 28 16:10:12.530163 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:12.530126    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" event={"ID":"89d15a04-9b9b-45c3-9dee-188d150229b7","Type":"ContainerDied","Data":"0f386a5704487e795c2c27e8ce2a8b1d7f0f367fc3d9a1c5708670f84a54520c"}
Jul 28 16:10:12.531778 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:12.531754    2575 generic.go:356] "Generic (PLEG): container finished" podID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerID="99b3c592a4671325805ddfc64683c30d5b564f90aa4b977a9ff416635e9bf230" exitCode=0
Jul 28 16:10:12.531875 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:12.531828    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" event={"ID":"3f237da2-64e5-4047-99e1-56bcce0f6e24","Type":"ContainerDied","Data":"99b3c592a4671325805ddfc64683c30d5b564f90aa4b977a9ff416635e9bf230"}
Jul 28 16:10:40.696625 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:40.696587    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" event={"ID":"a85b88d2-526f-4050-a7a4-84425d6173fd","Type":"ContainerStarted","Data":"7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4"}
Jul 28 16:10:41.708443 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:41.708399    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" event={"ID":"89d15a04-9b9b-45c3-9dee-188d150229b7","Type":"ContainerStarted","Data":"39da098506dc360df000fb04626d7d40af9d974381cc2ee8a4469126b1487b03"}
Jul 28 16:10:41.712892 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:41.712846    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" event={"ID":"3f237da2-64e5-4047-99e1-56bcce0f6e24","Type":"ContainerStarted","Data":"51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f"}
Jul 28 16:10:43.724569 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:43.724530    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" event={"ID":"a85b88d2-526f-4050-a7a4-84425d6173fd","Type":"ContainerStarted","Data":"caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93"}
Jul 28 16:10:43.725065 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:43.724873    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:43.726869 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:43.726843    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" event={"ID":"89d15a04-9b9b-45c3-9dee-188d150229b7","Type":"ContainerStarted","Data":"1fb3b969567bc017f4afc06b39f73d140158f2d8d7606241ae50974efd50981b"}
Jul 28 16:10:43.727457 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:43.727125    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:43.727457 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:43.727174    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:43.728515 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:43.728488    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 28 16:10:43.729440 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:43.729416    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" event={"ID":"3f237da2-64e5-4047-99e1-56bcce0f6e24","Type":"ContainerStarted","Data":"8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de"}
Jul 28 16:10:43.729545 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:43.729531    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:43.743385 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:43.743331    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podStartSLOduration=2.865296625 podStartE2EDuration="36.743314525s" totalImagesPullingTime="30.292148093s" totalInitContainerRuntime="3.585869807s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:10:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:10:43.743197019 +0000 UTC m=+569.887575571" watchObservedRunningTime="2026-07-28 16:10:43.743314525 +0000 UTC m=+569.887693078"
Jul 28 16:10:43.762507 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:43.762453    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podStartSLOduration=2.498774576 podStartE2EDuration="36.762435906s" totalImagesPullingTime="30.231887097s" totalInitContainerRuntime="4.031774233s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:10:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:10:43.761536875 +0000 UTC m=+569.905915430" watchObservedRunningTime="2026-07-28 16:10:43.762435906 +0000 UTC m=+569.906814458"
Jul 28 16:10:43.782746 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:43.782690    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" podStartSLOduration=2.6583439479999997 podStartE2EDuration="36.782672599s" totalImagesPullingTime="30.296988807s" totalInitContainerRuntime="3.827339844s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:10:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:10:43.78075785 +0000 UTC m=+569.925136402" watchObservedRunningTime="2026-07-28 16:10:43.782672599 +0000 UTC m=+569.927051151"
Jul 28 16:10:44.734863 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:44.734812    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 28 16:10:44.735472 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:44.734850    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:44.735586 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:44.735519    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:44.735770 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:44.735739    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 16:10:44.735885 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:44.735802    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 28 16:10:45.738135 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:45.738075    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 16:10:45.738643 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:45.738201    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 28 16:10:49.739367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:49.739334    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:10:49.739861 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:49.739834    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 28 16:10:50.743659 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:50.743631    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:10:50.744142 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:50.743718    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:10:50.744311 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:50.744281    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 16:10:50.744600 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:50.744572    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 28 16:10:59.739884 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:10:59.739840    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 28 16:11:00.744552 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:00.744506    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 16:11:00.744992 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:00.744506    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 28 16:11:09.739781 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:09.739744    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 28 16:11:10.744973 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:10.744932    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 28 16:11:10.744973 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:10.744960    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 16:11:19.740372 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:19.740326    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 28 16:11:20.745209 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:20.745169    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 28 16:11:20.745598 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:20.745169    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 16:11:29.740183 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:29.740134    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 28 16:11:30.744789 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:30.744747    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 28 16:11:30.745236 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:30.744748    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 16:11:39.739863 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:39.739775    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 28 16:11:40.744332 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:40.744282    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 16:11:40.744761 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:40.744446    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 28 16:11:49.741284 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:49.741243    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:11:50.744774 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:50.744729    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 28 16:11:50.745266 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:50.744729    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 28 16:11:59.439508 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:11:59.439452    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 28 16:12:00.744909 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:00.744878    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:12:09.441108 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:09.441067    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:12:51.181273 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:51.181238    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"]
Jul 28 16:12:51.181719 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:51.181499    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container" containerID="cri-o://7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4" gracePeriod=30
Jul 28 16:12:51.181719 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:51.181555    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kube-rbac-proxy" containerID="cri-o://caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93" gracePeriod=30
Jul 28 16:12:51.252236 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:51.252200    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"]
Jul 28 16:12:51.252491 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:51.252452    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container" containerID="cri-o://51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f" gracePeriod=30
Jul 28 16:12:51.252567 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:51.252487    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kube-rbac-proxy" containerID="cri-o://8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de" gracePeriod=30
Jul 28 16:12:51.325949 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:51.325916    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"]
Jul 28 16:12:51.326197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:51.326169    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kserve-container" containerID="cri-o://39da098506dc360df000fb04626d7d40af9d974381cc2ee8a4469126b1487b03" gracePeriod=30
Jul 28 16:12:51.326254 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:51.326215    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kube-rbac-proxy" containerID="cri-o://1fb3b969567bc017f4afc06b39f73d140158f2d8d7606241ae50974efd50981b" gracePeriod=30
Jul 28 16:12:52.228349 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:52.228305    2575 generic.go:356] "Generic (PLEG): container finished" podID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerID="caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93" exitCode=2
Jul 28 16:12:52.228845 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:52.228375    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" event={"ID":"a85b88d2-526f-4050-a7a4-84425d6173fd","Type":"ContainerDied","Data":"caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93"}
Jul 28 16:12:52.231031 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:52.231001    2575 generic.go:356] "Generic (PLEG): container finished" podID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerID="1fb3b969567bc017f4afc06b39f73d140158f2d8d7606241ae50974efd50981b" exitCode=2
Jul 28 16:12:52.231179 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:52.231072    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" event={"ID":"89d15a04-9b9b-45c3-9dee-188d150229b7","Type":"ContainerDied","Data":"1fb3b969567bc017f4afc06b39f73d140158f2d8d7606241ae50974efd50981b"}
Jul 28 16:12:52.233570 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:52.233550    2575 generic.go:356] "Generic (PLEG): container finished" podID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerID="8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de" exitCode=2
Jul 28 16:12:52.233674 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:52.233602    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" event={"ID":"3f237da2-64e5-4047-99e1-56bcce0f6e24","Type":"ContainerDied","Data":"8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de"}
Jul 28 16:12:54.735471 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:54.735421    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.20:8643/healthz\": dial tcp 10.133.0.20:8643: connect: connection refused"
Jul 28 16:12:55.248474 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.248434    2575 generic.go:356] "Generic (PLEG): container finished" podID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerID="39da098506dc360df000fb04626d7d40af9d974381cc2ee8a4469126b1487b03" exitCode=0
Jul 28 16:12:55.248646 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.248505    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" event={"ID":"89d15a04-9b9b-45c3-9dee-188d150229b7","Type":"ContainerDied","Data":"39da098506dc360df000fb04626d7d40af9d974381cc2ee8a4469126b1487b03"}
Jul 28 16:12:55.271504 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.271474    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:12:55.402034 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.401950    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/89d15a04-9b9b-45c3-9dee-188d150229b7-proxy-tls\") pod \"89d15a04-9b9b-45c3-9dee-188d150229b7\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") "
Jul 28 16:12:55.402218 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.402049    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/89d15a04-9b9b-45c3-9dee-188d150229b7-kube-rbac-proxy-sar-config\") pod \"89d15a04-9b9b-45c3-9dee-188d150229b7\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") "
Jul 28 16:12:55.402218 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.402071    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/89d15a04-9b9b-45c3-9dee-188d150229b7-kserve-provision-location\") pod \"89d15a04-9b9b-45c3-9dee-188d150229b7\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") "
Jul 28 16:12:55.402218 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.402089    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/89d15a04-9b9b-45c3-9dee-188d150229b7-cabundle-cert\") pod \"89d15a04-9b9b-45c3-9dee-188d150229b7\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") "
Jul 28 16:12:55.402218 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.402128    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blk6k\" (UniqueName: \"kubernetes.io/projected/89d15a04-9b9b-45c3-9dee-188d150229b7-kube-api-access-blk6k\") pod \"89d15a04-9b9b-45c3-9dee-188d150229b7\" (UID: \"89d15a04-9b9b-45c3-9dee-188d150229b7\") "
Jul 28 16:12:55.402442 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.402417    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89d15a04-9b9b-45c3-9dee-188d150229b7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "89d15a04-9b9b-45c3-9dee-188d150229b7" (UID: "89d15a04-9b9b-45c3-9dee-188d150229b7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 16:12:55.402490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.402426    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89d15a04-9b9b-45c3-9dee-188d150229b7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "89d15a04-9b9b-45c3-9dee-188d150229b7" (UID: "89d15a04-9b9b-45c3-9dee-188d150229b7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:12:55.402490 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.402434    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89d15a04-9b9b-45c3-9dee-188d150229b7-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "89d15a04-9b9b-45c3-9dee-188d150229b7" (UID: "89d15a04-9b9b-45c3-9dee-188d150229b7"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:12:55.404280 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.404251    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89d15a04-9b9b-45c3-9dee-188d150229b7-kube-api-access-blk6k" (OuterVolumeSpecName: "kube-api-access-blk6k") pod "89d15a04-9b9b-45c3-9dee-188d150229b7" (UID: "89d15a04-9b9b-45c3-9dee-188d150229b7"). InnerVolumeSpecName "kube-api-access-blk6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:12:55.404340 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.404263    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89d15a04-9b9b-45c3-9dee-188d150229b7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "89d15a04-9b9b-45c3-9dee-188d150229b7" (UID: "89d15a04-9b9b-45c3-9dee-188d150229b7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:12:55.503044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.502998    2575 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/89d15a04-9b9b-45c3-9dee-188d150229b7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:55.503044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.503035    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/89d15a04-9b9b-45c3-9dee-188d150229b7-kserve-provision-location\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:55.503044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.503049    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/89d15a04-9b9b-45c3-9dee-188d150229b7-cabundle-cert\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:55.503317 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.503062    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blk6k\" (UniqueName: \"kubernetes.io/projected/89d15a04-9b9b-45c3-9dee-188d150229b7-kube-api-access-blk6k\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:55.503317 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.503074    2575 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/89d15a04-9b9b-45c3-9dee-188d150229b7-proxy-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:55.738962 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.738920    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused"
Jul 28 16:12:55.739384 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:55.738920    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.19:8643/healthz\": dial tcp 10.133.0.19:8643: connect: connection refused"
Jul 28 16:12:56.254536 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.254502    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2" event={"ID":"89d15a04-9b9b-45c3-9dee-188d150229b7","Type":"ContainerDied","Data":"2e954cb92bb702bb63f9f9a2b78d98d018d61eec9d3f9cfc7205d6e35091cc88"}
Jul 28 16:12:56.254688 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.254553    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"
Jul 28 16:12:56.254688 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.254576    2575 scope.go:117] "RemoveContainer" containerID="1fb3b969567bc017f4afc06b39f73d140158f2d8d7606241ae50974efd50981b"
Jul 28 16:12:56.291925 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.291818    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"]
Jul 28 16:12:56.291987 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.291955    2575 scope.go:117] "RemoveContainer" containerID="39da098506dc360df000fb04626d7d40af9d974381cc2ee8a4469126b1487b03"
Jul 28 16:12:56.298769 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.298737    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6xnd2"]
Jul 28 16:12:56.300272 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.300248    2575 scope.go:117] "RemoveContainer" containerID="0f386a5704487e795c2c27e8ce2a8b1d7f0f367fc3d9a1c5708670f84a54520c"
Jul 28 16:12:56.422469 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.422444    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:12:56.444925 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.444893    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" path="/var/lib/kubelet/pods/89d15a04-9b9b-45c3-9dee-188d150229b7/volumes"
Jul 28 16:12:56.492631 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.492605    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:12:56.511505 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.511479    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a85b88d2-526f-4050-a7a4-84425d6173fd-proxy-tls\") pod \"a85b88d2-526f-4050-a7a4-84425d6173fd\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") "
Jul 28 16:12:56.511636 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.511542    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a85b88d2-526f-4050-a7a4-84425d6173fd-cabundle-cert\") pod \"a85b88d2-526f-4050-a7a4-84425d6173fd\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") "
Jul 28 16:12:56.511636 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.511576    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8xd4\" (UniqueName: \"kubernetes.io/projected/a85b88d2-526f-4050-a7a4-84425d6173fd-kube-api-access-f8xd4\") pod \"a85b88d2-526f-4050-a7a4-84425d6173fd\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") "
Jul 28 16:12:56.511636 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.511618    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a85b88d2-526f-4050-a7a4-84425d6173fd-kube-rbac-proxy-sar-config\") pod \"a85b88d2-526f-4050-a7a4-84425d6173fd\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") "
Jul 28 16:12:56.511795 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.511656    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a85b88d2-526f-4050-a7a4-84425d6173fd-kserve-provision-location\") pod \"a85b88d2-526f-4050-a7a4-84425d6173fd\" (UID: \"a85b88d2-526f-4050-a7a4-84425d6173fd\") "
Jul 28 16:12:56.512058 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.512026    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a85b88d2-526f-4050-a7a4-84425d6173fd-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "a85b88d2-526f-4050-a7a4-84425d6173fd" (UID: "a85b88d2-526f-4050-a7a4-84425d6173fd"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:12:56.512058 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.512034    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a85b88d2-526f-4050-a7a4-84425d6173fd-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a85b88d2-526f-4050-a7a4-84425d6173fd" (UID: "a85b88d2-526f-4050-a7a4-84425d6173fd"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:12:56.512249 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.512034    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a85b88d2-526f-4050-a7a4-84425d6173fd-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "a85b88d2-526f-4050-a7a4-84425d6173fd" (UID: "a85b88d2-526f-4050-a7a4-84425d6173fd"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 16:12:56.514030 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.514003    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a85b88d2-526f-4050-a7a4-84425d6173fd-kube-api-access-f8xd4" (OuterVolumeSpecName: "kube-api-access-f8xd4") pod "a85b88d2-526f-4050-a7a4-84425d6173fd" (UID: "a85b88d2-526f-4050-a7a4-84425d6173fd"). InnerVolumeSpecName "kube-api-access-f8xd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:12:56.514180 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.514135    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a85b88d2-526f-4050-a7a4-84425d6173fd-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a85b88d2-526f-4050-a7a4-84425d6173fd" (UID: "a85b88d2-526f-4050-a7a4-84425d6173fd"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:12:56.612845 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.612808    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f237da2-64e5-4047-99e1-56bcce0f6e24-kserve-provision-location\") pod \"3f237da2-64e5-4047-99e1-56bcce0f6e24\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") "
Jul 28 16:12:56.613017 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.612858    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f237da2-64e5-4047-99e1-56bcce0f6e24-kube-rbac-proxy-sar-config\") pod \"3f237da2-64e5-4047-99e1-56bcce0f6e24\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") "
Jul 28 16:12:56.613017 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.612879    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f237da2-64e5-4047-99e1-56bcce0f6e24-proxy-tls\") pod \"3f237da2-64e5-4047-99e1-56bcce0f6e24\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") "
Jul 28 16:12:56.613017 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.612918    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f237da2-64e5-4047-99e1-56bcce0f6e24-cabundle-cert\") pod \"3f237da2-64e5-4047-99e1-56bcce0f6e24\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") "
Jul 28 16:12:56.613017 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.612939    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdjfm\" (UniqueName: \"kubernetes.io/projected/3f237da2-64e5-4047-99e1-56bcce0f6e24-kube-api-access-cdjfm\") pod \"3f237da2-64e5-4047-99e1-56bcce0f6e24\" (UID: \"3f237da2-64e5-4047-99e1-56bcce0f6e24\") "
Jul 28 16:12:56.613276 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.613081    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8xd4\" (UniqueName: \"kubernetes.io/projected/a85b88d2-526f-4050-a7a4-84425d6173fd-kube-api-access-f8xd4\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:56.613276 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.613119    2575 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a85b88d2-526f-4050-a7a4-84425d6173fd-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:56.613276 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.613134    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a85b88d2-526f-4050-a7a4-84425d6173fd-kserve-provision-location\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:56.613276 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.613148    2575 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a85b88d2-526f-4050-a7a4-84425d6173fd-proxy-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:56.613276 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.613159    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a85b88d2-526f-4050-a7a4-84425d6173fd-cabundle-cert\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:56.613276 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.613268    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f237da2-64e5-4047-99e1-56bcce0f6e24-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3f237da2-64e5-4047-99e1-56bcce0f6e24" (UID: "3f237da2-64e5-4047-99e1-56bcce0f6e24"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:12:56.613500 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.613266    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f237da2-64e5-4047-99e1-56bcce0f6e24-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "3f237da2-64e5-4047-99e1-56bcce0f6e24" (UID: "3f237da2-64e5-4047-99e1-56bcce0f6e24"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 16:12:56.613500 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.613352    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f237da2-64e5-4047-99e1-56bcce0f6e24-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "3f237da2-64e5-4047-99e1-56bcce0f6e24" (UID: "3f237da2-64e5-4047-99e1-56bcce0f6e24"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:12:56.615023 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.615003    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f237da2-64e5-4047-99e1-56bcce0f6e24-kube-api-access-cdjfm" (OuterVolumeSpecName: "kube-api-access-cdjfm") pod "3f237da2-64e5-4047-99e1-56bcce0f6e24" (UID: "3f237da2-64e5-4047-99e1-56bcce0f6e24"). InnerVolumeSpecName "kube-api-access-cdjfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:12:56.615077 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.615025    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f237da2-64e5-4047-99e1-56bcce0f6e24-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3f237da2-64e5-4047-99e1-56bcce0f6e24" (UID: "3f237da2-64e5-4047-99e1-56bcce0f6e24"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:12:56.713522 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.713489    2575 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f237da2-64e5-4047-99e1-56bcce0f6e24-kserve-provision-location\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:56.713522 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.713517    2575 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f237da2-64e5-4047-99e1-56bcce0f6e24-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:56.713522 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.713527    2575 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f237da2-64e5-4047-99e1-56bcce0f6e24-proxy-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:56.713522 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.713536    2575 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f237da2-64e5-4047-99e1-56bcce0f6e24-cabundle-cert\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:56.713819 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:56.713545    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdjfm\" (UniqueName: \"kubernetes.io/projected/3f237da2-64e5-4047-99e1-56bcce0f6e24-kube-api-access-cdjfm\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:12:57.260016 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.259970    2575 generic.go:356] "Generic (PLEG): container finished" podID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerID="51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f" exitCode=0
Jul 28 16:12:57.260473 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.260031    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"
Jul 28 16:12:57.260473 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.260050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" event={"ID":"3f237da2-64e5-4047-99e1-56bcce0f6e24","Type":"ContainerDied","Data":"51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f"}
Jul 28 16:12:57.260473 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.260142    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7" event={"ID":"3f237da2-64e5-4047-99e1-56bcce0f6e24","Type":"ContainerDied","Data":"86e6a391d8430e9929b24a4c5c4f1d060ca077404f2f3b4b8c13036dfce13287"}
Jul 28 16:12:57.260473 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.260183    2575 scope.go:117] "RemoveContainer" containerID="8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de"
Jul 28 16:12:57.262084 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.262060    2575 generic.go:356] "Generic (PLEG): container finished" podID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerID="7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4" exitCode=0
Jul 28 16:12:57.262200 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.262137    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" event={"ID":"a85b88d2-526f-4050-a7a4-84425d6173fd","Type":"ContainerDied","Data":"7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4"}
Jul 28 16:12:57.262264 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.262196    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"
Jul 28 16:12:57.262264 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.262210    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk" event={"ID":"a85b88d2-526f-4050-a7a4-84425d6173fd","Type":"ContainerDied","Data":"19177cbf4d21aeb11049b454c511be672003f6153c2987e8b4917c68acb1a266"}
Jul 28 16:12:57.270061 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.270037    2575 scope.go:117] "RemoveContainer" containerID="51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f"
Jul 28 16:12:57.278137 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.278113    2575 scope.go:117] "RemoveContainer" containerID="99b3c592a4671325805ddfc64683c30d5b564f90aa4b977a9ff416635e9bf230"
Jul 28 16:12:57.284918 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.284886    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"]
Jul 28 16:12:57.286534 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.286514    2575 scope.go:117] "RemoveContainer" containerID="8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de"
Jul 28 16:12:57.286858 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:12:57.286839    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de\": container with ID starting with 8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de not found: ID does not exist" containerID="8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de"
Jul 28 16:12:57.286905 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.286870    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de"} err="failed to get container status \"8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de\": rpc error: code = NotFound desc = could not find container \"8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de\": container with ID starting with 8c51c7f7e609c5c6da3520fa3b53462be7cfa18cb922d402f8a5b7e71fd5b4de not found: ID does not exist"
Jul 28 16:12:57.286905 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.286893    2575 scope.go:117] "RemoveContainer" containerID="51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f"
Jul 28 16:12:57.287199 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:12:57.287176    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f\": container with ID starting with 51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f not found: ID does not exist" containerID="51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f"
Jul 28 16:12:57.287277 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.287208    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f"} err="failed to get container status \"51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f\": rpc error: code = NotFound desc = could not find container \"51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f\": container with ID starting with 51321c86641e84ba93887a08bb29dd65e6e2fb61703aa00ae4e95035ecfd2b0f not found: ID does not exist"
Jul 28 16:12:57.287277 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.287230    2575 scope.go:117] "RemoveContainer" containerID="99b3c592a4671325805ddfc64683c30d5b564f90aa4b977a9ff416635e9bf230"
Jul 28 16:12:57.287465 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:12:57.287450    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99b3c592a4671325805ddfc64683c30d5b564f90aa4b977a9ff416635e9bf230\": container with ID starting with 99b3c592a4671325805ddfc64683c30d5b564f90aa4b977a9ff416635e9bf230 not found: ID does not exist" containerID="99b3c592a4671325805ddfc64683c30d5b564f90aa4b977a9ff416635e9bf230"
Jul 28 16:12:57.287506 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.287469    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99b3c592a4671325805ddfc64683c30d5b564f90aa4b977a9ff416635e9bf230"} err="failed to get container status \"99b3c592a4671325805ddfc64683c30d5b564f90aa4b977a9ff416635e9bf230\": rpc error: code = NotFound desc = could not find container \"99b3c592a4671325805ddfc64683c30d5b564f90aa4b977a9ff416635e9bf230\": container with ID starting with 99b3c592a4671325805ddfc64683c30d5b564f90aa4b977a9ff416635e9bf230 not found: ID does not exist"
Jul 28 16:12:57.287506 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.287484    2575 scope.go:117] "RemoveContainer" containerID="caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93"
Jul 28 16:12:57.291715 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.291690    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-84c58bd689-44kr7"]
Jul 28 16:12:57.295583 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.295566    2575 scope.go:117] "RemoveContainer" containerID="7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4"
Jul 28 16:12:57.303355 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.303329    2575 scope.go:117] "RemoveContainer" containerID="c855a2db6ae3fc9d913bcd29629574885d42accc0b7eb115599e5e5ec35809c7"
Jul 28 16:12:57.305987 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.305962    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"]
Jul 28 16:12:57.310617 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.310592    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5bc9545b47-wz2bk"]
Jul 28 16:12:57.312459 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.312442    2575 scope.go:117] "RemoveContainer" containerID="caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93"
Jul 28 16:12:57.312768 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:12:57.312745    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93\": container with ID starting with caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93 not found: ID does not exist" containerID="caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93"
Jul 28 16:12:57.312836 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.312777    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93"} err="failed to get container status \"caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93\": rpc error: code = NotFound desc = could not find container \"caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93\": container with ID starting with caee86409e2e9ab93999fe5b16ed0dc9c41c497d9fb77ac30262a80cb4252a93 not found: ID does not exist"
Jul 28 16:12:57.312836 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.312796    2575 scope.go:117] "RemoveContainer" containerID="7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4"
Jul 28 16:12:57.313041 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:12:57.313025    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4\": container with ID starting with 7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4 not found: ID does not exist" containerID="7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4"
Jul 28 16:12:57.313081 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.313048    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4"} err="failed to get container status \"7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4\": rpc error: code = NotFound desc = could not find container \"7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4\": container with ID starting with 7d9a3a3cc0fe0e74908d378e31a3563756674faecccf9c5da812638dc42943e4 not found: ID does not exist"
Jul 28 16:12:57.313081 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.313064    2575 scope.go:117] "RemoveContainer" containerID="c855a2db6ae3fc9d913bcd29629574885d42accc0b7eb115599e5e5ec35809c7"
Jul 28 16:12:57.313276 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:12:57.313262    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c855a2db6ae3fc9d913bcd29629574885d42accc0b7eb115599e5e5ec35809c7\": container with ID starting with c855a2db6ae3fc9d913bcd29629574885d42accc0b7eb115599e5e5ec35809c7 not found: ID does not exist" containerID="c855a2db6ae3fc9d913bcd29629574885d42accc0b7eb115599e5e5ec35809c7"
Jul 28 16:12:57.313323 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:57.313280    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c855a2db6ae3fc9d913bcd29629574885d42accc0b7eb115599e5e5ec35809c7"} err="failed to get container status \"c855a2db6ae3fc9d913bcd29629574885d42accc0b7eb115599e5e5ec35809c7\": rpc error: code = NotFound desc = could not find container \"c855a2db6ae3fc9d913bcd29629574885d42accc0b7eb115599e5e5ec35809c7\": container with ID starting with c855a2db6ae3fc9d913bcd29629574885d42accc0b7eb115599e5e5ec35809c7 not found: ID does not exist"
Jul 28 16:12:58.443931 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:58.443896    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" path="/var/lib/kubelet/pods/3f237da2-64e5-4047-99e1-56bcce0f6e24/volumes"
Jul 28 16:12:58.444416 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:12:58.444401    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" path="/var/lib/kubelet/pods/a85b88d2-526f-4050-a7a4-84425d6173fd/volumes"
Jul 28 16:13:51.317575 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317539    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"]
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317798    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317810    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317821    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="storage-initializer"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317827    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="storage-initializer"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317833    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kserve-container"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317839    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kserve-container"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317846    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kube-rbac-proxy"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317851    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kube-rbac-proxy"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317858    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kube-rbac-proxy"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317865    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kube-rbac-proxy"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317872    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kube-rbac-proxy"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317877    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kube-rbac-proxy"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317885    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317890    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317896    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="storage-initializer"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317902    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="storage-initializer"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317907    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="storage-initializer"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317912    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="storage-initializer"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317954    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kube-rbac-proxy"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317961    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kserve-container"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317968    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kserve-container"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317974    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f237da2-64e5-4047-99e1-56bcce0f6e24" containerName="kube-rbac-proxy"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317979    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a85b88d2-526f-4050-a7a4-84425d6173fd" containerName="kserve-container"
Jul 28 16:13:51.318059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.317984    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="89d15a04-9b9b-45c3-9dee-188d150229b7" containerName="kube-rbac-proxy"
Jul 28 16:13:51.319848 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.319815    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:13:51.323197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.323174    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 28 16:13:51.323321 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.323173    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-k6dfd\""
Jul 28 16:13:51.323511 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.323492    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-0402e-kube-rbac-proxy-sar-config\""
Jul 28 16:13:51.323511 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.323502    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-0402e-serving-cert\""
Jul 28 16:13:51.331857 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.331833    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"]
Jul 28 16:13:51.443125 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.443062    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67851d84-2145-4769-ab2e-79d73df863a6-openshift-service-ca-bundle\") pod \"sequence-graph-0402e-5f87ffd8fd-xs84s\" (UID: \"67851d84-2145-4769-ab2e-79d73df863a6\") " pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:13:51.443318 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.443208    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/67851d84-2145-4769-ab2e-79d73df863a6-proxy-tls\") pod \"sequence-graph-0402e-5f87ffd8fd-xs84s\" (UID: \"67851d84-2145-4769-ab2e-79d73df863a6\") " pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:13:51.544591 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.544548    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67851d84-2145-4769-ab2e-79d73df863a6-openshift-service-ca-bundle\") pod \"sequence-graph-0402e-5f87ffd8fd-xs84s\" (UID: \"67851d84-2145-4769-ab2e-79d73df863a6\") " pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:13:51.544771 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.544625    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/67851d84-2145-4769-ab2e-79d73df863a6-proxy-tls\") pod \"sequence-graph-0402e-5f87ffd8fd-xs84s\" (UID: \"67851d84-2145-4769-ab2e-79d73df863a6\") " pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:13:51.545227 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.545206    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67851d84-2145-4769-ab2e-79d73df863a6-openshift-service-ca-bundle\") pod \"sequence-graph-0402e-5f87ffd8fd-xs84s\" (UID: \"67851d84-2145-4769-ab2e-79d73df863a6\") " pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:13:51.547052 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.547027    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/67851d84-2145-4769-ab2e-79d73df863a6-proxy-tls\") pod \"sequence-graph-0402e-5f87ffd8fd-xs84s\" (UID: \"67851d84-2145-4769-ab2e-79d73df863a6\") " pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:13:51.629918 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.629825    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:13:51.764472 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.764448    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"]
Jul 28 16:13:51.767045 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:13:51.767020    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67851d84_2145_4769_ab2e_79d73df863a6.slice/crio-6eb50550a3ff9e7118f99349bab0e35f7875a7238ba7b9597e3f65500704739a WatchSource:0}: Error finding container 6eb50550a3ff9e7118f99349bab0e35f7875a7238ba7b9597e3f65500704739a: Status 404 returned error can't find the container with id 6eb50550a3ff9e7118f99349bab0e35f7875a7238ba7b9597e3f65500704739a
Jul 28 16:13:51.769087 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:51.769072    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 16:13:52.464888 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:52.464848    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s" event={"ID":"67851d84-2145-4769-ab2e-79d73df863a6","Type":"ContainerStarted","Data":"6eb50550a3ff9e7118f99349bab0e35f7875a7238ba7b9597e3f65500704739a"}
Jul 28 16:13:54.474041 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:54.473998    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s" event={"ID":"67851d84-2145-4769-ab2e-79d73df863a6","Type":"ContainerStarted","Data":"66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae"}
Jul 28 16:13:54.474518 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:54.474133    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:13:54.493733 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:13:54.493679    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s" podStartSLOduration=1.42709693 podStartE2EDuration="3.493662208s" totalImagesPullingTime="2.066565278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:13:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:13:54.491691227 +0000 UTC m=+760.636069785" watchObservedRunningTime="2026-07-28 16:13:54.493662208 +0000 UTC m=+760.638040812"
Jul 28 16:14:00.484025 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:14:00.483991    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:22:05.911620 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:05.911582    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"]
Jul 28 16:22:05.912118 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:05.911815    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s" podUID="67851d84-2145-4769-ab2e-79d73df863a6" containerName="sequence-graph-0402e" containerID="cri-o://66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae" gracePeriod=30
Jul 28 16:22:06.189791 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:06.189704    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"]
Jul 28 16:22:06.192937 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:06.192918    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:06.195270 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:06.195242    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-daf0b-serving-cert\""
Jul 28 16:22:06.195397 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:06.195244    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-daf0b-kube-rbac-proxy-sar-config\""
Jul 28 16:22:06.203964 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:06.203931    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"]
Jul 28 16:22:06.245683 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:06.245635    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b26ca8c-1441-43d1-a838-300c59815740-openshift-service-ca-bundle\") pod \"ensemble-graph-daf0b-8f788765b-8f4h9\" (UID: \"3b26ca8c-1441-43d1-a838-300c59815740\") " pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:06.245683 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:06.245686    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b26ca8c-1441-43d1-a838-300c59815740-proxy-tls\") pod \"ensemble-graph-daf0b-8f788765b-8f4h9\" (UID: \"3b26ca8c-1441-43d1-a838-300c59815740\") " pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:06.347005 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:06.346964    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b26ca8c-1441-43d1-a838-300c59815740-openshift-service-ca-bundle\") pod \"ensemble-graph-daf0b-8f788765b-8f4h9\" (UID: \"3b26ca8c-1441-43d1-a838-300c59815740\") " pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:06.347005 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:06.347013    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b26ca8c-1441-43d1-a838-300c59815740-proxy-tls\") pod \"ensemble-graph-daf0b-8f788765b-8f4h9\" (UID: \"3b26ca8c-1441-43d1-a838-300c59815740\") " pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:06.347299 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:22:06.347150    2575 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-daf0b-serving-cert: secret "ensemble-graph-daf0b-serving-cert" not found
Jul 28 16:22:06.347299 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:22:06.347207    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b26ca8c-1441-43d1-a838-300c59815740-proxy-tls podName:3b26ca8c-1441-43d1-a838-300c59815740 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.847190525 +0000 UTC m=+1252.991569056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3b26ca8c-1441-43d1-a838-300c59815740-proxy-tls") pod "ensemble-graph-daf0b-8f788765b-8f4h9" (UID: "3b26ca8c-1441-43d1-a838-300c59815740") : secret "ensemble-graph-daf0b-serving-cert" not found
Jul 28 16:22:06.347738 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:06.347714    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b26ca8c-1441-43d1-a838-300c59815740-openshift-service-ca-bundle\") pod \"ensemble-graph-daf0b-8f788765b-8f4h9\" (UID: \"3b26ca8c-1441-43d1-a838-300c59815740\") " pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:06.852129 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:06.852056    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b26ca8c-1441-43d1-a838-300c59815740-proxy-tls\") pod \"ensemble-graph-daf0b-8f788765b-8f4h9\" (UID: \"3b26ca8c-1441-43d1-a838-300c59815740\") " pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:06.854726 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:06.854680    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b26ca8c-1441-43d1-a838-300c59815740-proxy-tls\") pod \"ensemble-graph-daf0b-8f788765b-8f4h9\" (UID: \"3b26ca8c-1441-43d1-a838-300c59815740\") " pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:07.103980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:07.103888    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:07.245525 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:07.243213    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"]
Jul 28 16:22:08.252989 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:08.252950    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9" event={"ID":"3b26ca8c-1441-43d1-a838-300c59815740","Type":"ContainerStarted","Data":"fdf8c0a82518873c3f309e62e6d3090527bb1c3aeb13d9c27a9c00909a22761d"}
Jul 28 16:22:08.253429 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:08.253015    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9" event={"ID":"3b26ca8c-1441-43d1-a838-300c59815740","Type":"ContainerStarted","Data":"800d6a58118b8712a18226d77f377c52a865dd33ae5e06d2a338185ea8b0799f"}
Jul 28 16:22:08.253429 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:08.253080    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:08.275333 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:08.275285    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9" podStartSLOduration=2.27526648 podStartE2EDuration="2.27526648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:08.275017415 +0000 UTC m=+1254.419396195" watchObservedRunningTime="2026-07-28 16:22:08.27526648 +0000 UTC m=+1254.419645032"
Jul 28 16:22:10.481690 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:10.481641    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s" podUID="67851d84-2145-4769-ab2e-79d73df863a6" containerName="sequence-graph-0402e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:14.264311 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:14.264278    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:15.481721 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:15.481681    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s" podUID="67851d84-2145-4769-ab2e-79d73df863a6" containerName="sequence-graph-0402e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:16.204732 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:16.204695    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"]
Jul 28 16:22:16.204899 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:16.204872    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9" podUID="3b26ca8c-1441-43d1-a838-300c59815740" containerName="ensemble-graph-daf0b" containerID="cri-o://fdf8c0a82518873c3f309e62e6d3090527bb1c3aeb13d9c27a9c00909a22761d" gracePeriod=30
Jul 28 16:22:19.261329 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:19.261286    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9" podUID="3b26ca8c-1441-43d1-a838-300c59815740" containerName="ensemble-graph-daf0b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:20.482829 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:20.482787    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s" podUID="67851d84-2145-4769-ab2e-79d73df863a6" containerName="sequence-graph-0402e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:20.483393 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:20.482894    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:22:24.261420 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:24.261377    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9" podUID="3b26ca8c-1441-43d1-a838-300c59815740" containerName="ensemble-graph-daf0b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:25.481787 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:25.481749    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s" podUID="67851d84-2145-4769-ab2e-79d73df863a6" containerName="sequence-graph-0402e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:29.261325 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:29.261284    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9" podUID="3b26ca8c-1441-43d1-a838-300c59815740" containerName="ensemble-graph-daf0b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:29.261707 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:29.261396    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:30.481921 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:30.481883    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s" podUID="67851d84-2145-4769-ab2e-79d73df863a6" containerName="sequence-graph-0402e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:34.260959 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:34.260919    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9" podUID="3b26ca8c-1441-43d1-a838-300c59815740" containerName="ensemble-graph-daf0b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:35.482579 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:35.482541    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s" podUID="67851d84-2145-4769-ab2e-79d73df863a6" containerName="sequence-graph-0402e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:36.059221 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.059189    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:22:36.190721 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.190614    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67851d84-2145-4769-ab2e-79d73df863a6-openshift-service-ca-bundle\") pod \"67851d84-2145-4769-ab2e-79d73df863a6\" (UID: \"67851d84-2145-4769-ab2e-79d73df863a6\") "
Jul 28 16:22:36.190721 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.190698    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/67851d84-2145-4769-ab2e-79d73df863a6-proxy-tls\") pod \"67851d84-2145-4769-ab2e-79d73df863a6\" (UID: \"67851d84-2145-4769-ab2e-79d73df863a6\") "
Jul 28 16:22:36.191002 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.190981    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67851d84-2145-4769-ab2e-79d73df863a6-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "67851d84-2145-4769-ab2e-79d73df863a6" (UID: "67851d84-2145-4769-ab2e-79d73df863a6"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:22:36.192744 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.192721    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67851d84-2145-4769-ab2e-79d73df863a6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "67851d84-2145-4769-ab2e-79d73df863a6" (UID: "67851d84-2145-4769-ab2e-79d73df863a6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:22:36.291450 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.291409    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67851d84-2145-4769-ab2e-79d73df863a6-openshift-service-ca-bundle\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:22:36.291450 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.291445    2575 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/67851d84-2145-4769-ab2e-79d73df863a6-proxy-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:22:36.357685 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.357644    2575 generic.go:356] "Generic (PLEG): container finished" podID="67851d84-2145-4769-ab2e-79d73df863a6" containerID="66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae" exitCode=0
Jul 28 16:22:36.357685 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.357690    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"
Jul 28 16:22:36.357858 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.357709    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s" event={"ID":"67851d84-2145-4769-ab2e-79d73df863a6","Type":"ContainerDied","Data":"66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae"}
Jul 28 16:22:36.357858 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.357763    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s" event={"ID":"67851d84-2145-4769-ab2e-79d73df863a6","Type":"ContainerDied","Data":"6eb50550a3ff9e7118f99349bab0e35f7875a7238ba7b9597e3f65500704739a"}
Jul 28 16:22:36.357858 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.357798    2575 scope.go:117] "RemoveContainer" containerID="66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae"
Jul 28 16:22:36.366940 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.366917    2575 scope.go:117] "RemoveContainer" containerID="66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae"
Jul 28 16:22:36.367281 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:22:36.367260    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae\": container with ID starting with 66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae not found: ID does not exist" containerID="66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae"
Jul 28 16:22:36.367339 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.367290    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae"} err="failed to get container status \"66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae\": rpc error: code = NotFound desc = could not find container \"66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae\": container with ID starting with 66cca3aca9a4dee590baa4e3714e0327adf7ca13acedef5b62a68c6436524aae not found: ID does not exist"
Jul 28 16:22:36.378473 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.378436    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"]
Jul 28 16:22:36.382032 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.382005    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-0402e-5f87ffd8fd-xs84s"]
Jul 28 16:22:36.443337 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:36.443254    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67851d84-2145-4769-ab2e-79d73df863a6" path="/var/lib/kubelet/pods/67851d84-2145-4769-ab2e-79d73df863a6/volumes"
Jul 28 16:22:39.261005 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:39.260965    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9" podUID="3b26ca8c-1441-43d1-a838-300c59815740" containerName="ensemble-graph-daf0b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:44.261837 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:44.261797    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9" podUID="3b26ca8c-1441-43d1-a838-300c59815740" containerName="ensemble-graph-daf0b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:46.403285 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:46.403247    2575 generic.go:356] "Generic (PLEG): container finished" podID="3b26ca8c-1441-43d1-a838-300c59815740" containerID="fdf8c0a82518873c3f309e62e6d3090527bb1c3aeb13d9c27a9c00909a22761d" exitCode=0
Jul 28 16:22:46.403664 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:46.403318    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9" event={"ID":"3b26ca8c-1441-43d1-a838-300c59815740","Type":"ContainerDied","Data":"fdf8c0a82518873c3f309e62e6d3090527bb1c3aeb13d9c27a9c00909a22761d"}
Jul 28 16:22:46.859955 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:46.859924    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:46.984226 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:46.984116    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b26ca8c-1441-43d1-a838-300c59815740-openshift-service-ca-bundle\") pod \"3b26ca8c-1441-43d1-a838-300c59815740\" (UID: \"3b26ca8c-1441-43d1-a838-300c59815740\") "
Jul 28 16:22:46.984226 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:46.984203    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b26ca8c-1441-43d1-a838-300c59815740-proxy-tls\") pod \"3b26ca8c-1441-43d1-a838-300c59815740\" (UID: \"3b26ca8c-1441-43d1-a838-300c59815740\") "
Jul 28 16:22:46.984568 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:46.984536    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b26ca8c-1441-43d1-a838-300c59815740-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "3b26ca8c-1441-43d1-a838-300c59815740" (UID: "3b26ca8c-1441-43d1-a838-300c59815740"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:22:46.986284 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:46.986259    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b26ca8c-1441-43d1-a838-300c59815740-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3b26ca8c-1441-43d1-a838-300c59815740" (UID: "3b26ca8c-1441-43d1-a838-300c59815740"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:22:47.084733 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:47.084694    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b26ca8c-1441-43d1-a838-300c59815740-openshift-service-ca-bundle\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:22:47.084733 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:47.084726    2575 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b26ca8c-1441-43d1-a838-300c59815740-proxy-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:22:47.408447 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:47.408361    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"
Jul 28 16:22:47.408909 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:47.408356    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9" event={"ID":"3b26ca8c-1441-43d1-a838-300c59815740","Type":"ContainerDied","Data":"800d6a58118b8712a18226d77f377c52a865dd33ae5e06d2a338185ea8b0799f"}
Jul 28 16:22:47.408909 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:47.408518    2575 scope.go:117] "RemoveContainer" containerID="fdf8c0a82518873c3f309e62e6d3090527bb1c3aeb13d9c27a9c00909a22761d"
Jul 28 16:22:47.430239 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:47.430198    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"]
Jul 28 16:22:47.434395 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:47.434364    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-daf0b-8f788765b-8f4h9"]
Jul 28 16:22:48.443381 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:22:48.443348    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b26ca8c-1441-43d1-a838-300c59815740" path="/var/lib/kubelet/pods/3b26ca8c-1441-43d1-a838-300c59815740/volumes"
Jul 28 16:23:16.146827 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.146784    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"]
Jul 28 16:23:16.147370 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.147172    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67851d84-2145-4769-ab2e-79d73df863a6" containerName="sequence-graph-0402e"
Jul 28 16:23:16.147370 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.147189    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="67851d84-2145-4769-ab2e-79d73df863a6" containerName="sequence-graph-0402e"
Jul 28 16:23:16.147370 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.147204    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b26ca8c-1441-43d1-a838-300c59815740" containerName="ensemble-graph-daf0b"
Jul 28 16:23:16.147370 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.147211    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b26ca8c-1441-43d1-a838-300c59815740" containerName="ensemble-graph-daf0b"
Jul 28 16:23:16.147370 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.147289    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b26ca8c-1441-43d1-a838-300c59815740" containerName="ensemble-graph-daf0b"
Jul 28 16:23:16.147370 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.147301    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="67851d84-2145-4769-ab2e-79d73df863a6" containerName="sequence-graph-0402e"
Jul 28 16:23:16.151466 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.151443    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:16.154347 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.154321    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-k6dfd\""
Jul 28 16:23:16.154551 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.154535    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-7c1b1-kube-rbac-proxy-sar-config\""
Jul 28 16:23:16.154599 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.154573    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 28 16:23:16.155912 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.155897    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-7c1b1-serving-cert\""
Jul 28 16:23:16.159733 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.159704    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"]
Jul 28 16:23:16.199552 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.199513    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f7340e-bc02-4848-8e61-5e66fa3d99f1-openshift-service-ca-bundle\") pod \"sequence-graph-7c1b1-5f49957547-pnznt\" (UID: \"00f7340e-bc02-4848-8e61-5e66fa3d99f1\") " pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:16.199732 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.199568    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00f7340e-bc02-4848-8e61-5e66fa3d99f1-proxy-tls\") pod \"sequence-graph-7c1b1-5f49957547-pnznt\" (UID: \"00f7340e-bc02-4848-8e61-5e66fa3d99f1\") " pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:16.300852 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.300813    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f7340e-bc02-4848-8e61-5e66fa3d99f1-openshift-service-ca-bundle\") pod \"sequence-graph-7c1b1-5f49957547-pnznt\" (UID: \"00f7340e-bc02-4848-8e61-5e66fa3d99f1\") " pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:16.301039 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.300880    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00f7340e-bc02-4848-8e61-5e66fa3d99f1-proxy-tls\") pod \"sequence-graph-7c1b1-5f49957547-pnznt\" (UID: \"00f7340e-bc02-4848-8e61-5e66fa3d99f1\") " pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:16.301039 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:23:16.300996    2575 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-7c1b1-serving-cert: secret "sequence-graph-7c1b1-serving-cert" not found
Jul 28 16:23:16.301158 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:23:16.301065    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00f7340e-bc02-4848-8e61-5e66fa3d99f1-proxy-tls podName:00f7340e-bc02-4848-8e61-5e66fa3d99f1 nodeName:}" failed. No retries permitted until 2026-07-28 16:23:16.801042531 +0000 UTC m=+1322.945421061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/00f7340e-bc02-4848-8e61-5e66fa3d99f1-proxy-tls") pod "sequence-graph-7c1b1-5f49957547-pnznt" (UID: "00f7340e-bc02-4848-8e61-5e66fa3d99f1") : secret "sequence-graph-7c1b1-serving-cert" not found
Jul 28 16:23:16.301614 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.301593    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f7340e-bc02-4848-8e61-5e66fa3d99f1-openshift-service-ca-bundle\") pod \"sequence-graph-7c1b1-5f49957547-pnznt\" (UID: \"00f7340e-bc02-4848-8e61-5e66fa3d99f1\") " pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:16.804426 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.804379    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00f7340e-bc02-4848-8e61-5e66fa3d99f1-proxy-tls\") pod \"sequence-graph-7c1b1-5f49957547-pnznt\" (UID: \"00f7340e-bc02-4848-8e61-5e66fa3d99f1\") " pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:16.806965 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:16.806937    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00f7340e-bc02-4848-8e61-5e66fa3d99f1-proxy-tls\") pod \"sequence-graph-7c1b1-5f49957547-pnznt\" (UID: \"00f7340e-bc02-4848-8e61-5e66fa3d99f1\") " pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:17.061916 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:17.061828    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:17.200833 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:17.200798    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"]
Jul 28 16:23:17.204153 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:23:17.204124    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00f7340e_bc02_4848_8e61_5e66fa3d99f1.slice/crio-ed11a4c0ee1bc4f28619ede73029b7f9c5bbaad4723c29966a3723f673bb9ea0 WatchSource:0}: Error finding container ed11a4c0ee1bc4f28619ede73029b7f9c5bbaad4723c29966a3723f673bb9ea0: Status 404 returned error can't find the container with id ed11a4c0ee1bc4f28619ede73029b7f9c5bbaad4723c29966a3723f673bb9ea0
Jul 28 16:23:17.520600 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:17.520562    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt" event={"ID":"00f7340e-bc02-4848-8e61-5e66fa3d99f1","Type":"ContainerStarted","Data":"af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b"}
Jul 28 16:23:17.520779 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:17.520613    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt" event={"ID":"00f7340e-bc02-4848-8e61-5e66fa3d99f1","Type":"ContainerStarted","Data":"ed11a4c0ee1bc4f28619ede73029b7f9c5bbaad4723c29966a3723f673bb9ea0"}
Jul 28 16:23:17.520779 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:17.520640    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:17.539559 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:17.539504    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt" podStartSLOduration=1.539484382 podStartE2EDuration="1.539484382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:23:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:23:17.537727567 +0000 UTC m=+1323.682106139" watchObservedRunningTime="2026-07-28 16:23:17.539484382 +0000 UTC m=+1323.683862935"
Jul 28 16:23:23.530521 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:23.530492    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:26.205027 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.204991    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"]
Jul 28 16:23:26.205498 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.205223    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt" podUID="00f7340e-bc02-4848-8e61-5e66fa3d99f1" containerName="sequence-graph-7c1b1" containerID="cri-o://af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b" gracePeriod=30
Jul 28 16:23:26.529943 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.529906    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"]
Jul 28 16:23:26.533376 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.533354    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:23:26.535738 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.535715    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-e5aa9-kube-rbac-proxy-sar-config\""
Jul 28 16:23:26.535817 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.535728    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-e5aa9-serving-cert\""
Jul 28 16:23:26.546731 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.546694    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"]
Jul 28 16:23:26.580679 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.580646    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee779df5-c5a9-4762-9113-f8c2ebd538e4-proxy-tls\") pod \"ensemble-graph-e5aa9-657946d8cb-xvlqc\" (UID: \"ee779df5-c5a9-4762-9113-f8c2ebd538e4\") " pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:23:26.580679 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.580686    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee779df5-c5a9-4762-9113-f8c2ebd538e4-openshift-service-ca-bundle\") pod \"ensemble-graph-e5aa9-657946d8cb-xvlqc\" (UID: \"ee779df5-c5a9-4762-9113-f8c2ebd538e4\") " pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:23:26.681722 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.681683    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee779df5-c5a9-4762-9113-f8c2ebd538e4-proxy-tls\") pod \"ensemble-graph-e5aa9-657946d8cb-xvlqc\" (UID: \"ee779df5-c5a9-4762-9113-f8c2ebd538e4\") " pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:23:26.681722 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.681737    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee779df5-c5a9-4762-9113-f8c2ebd538e4-openshift-service-ca-bundle\") pod \"ensemble-graph-e5aa9-657946d8cb-xvlqc\" (UID: \"ee779df5-c5a9-4762-9113-f8c2ebd538e4\") " pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:23:26.682479 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.682452    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee779df5-c5a9-4762-9113-f8c2ebd538e4-openshift-service-ca-bundle\") pod \"ensemble-graph-e5aa9-657946d8cb-xvlqc\" (UID: \"ee779df5-c5a9-4762-9113-f8c2ebd538e4\") " pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:23:26.684296 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.684275    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee779df5-c5a9-4762-9113-f8c2ebd538e4-proxy-tls\") pod \"ensemble-graph-e5aa9-657946d8cb-xvlqc\" (UID: \"ee779df5-c5a9-4762-9113-f8c2ebd538e4\") " pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:23:26.844291 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.844191    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:23:26.983204 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:26.983169    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"]
Jul 28 16:23:26.986704 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:23:26.986669    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee779df5_c5a9_4762_9113_f8c2ebd538e4.slice/crio-3931286eb1d09e0cc05cae986a46464505e62b4515b6b7863bcdbed114eb5896 WatchSource:0}: Error finding container 3931286eb1d09e0cc05cae986a46464505e62b4515b6b7863bcdbed114eb5896: Status 404 returned error can't find the container with id 3931286eb1d09e0cc05cae986a46464505e62b4515b6b7863bcdbed114eb5896
Jul 28 16:23:27.555938 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:27.555898    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc" event={"ID":"ee779df5-c5a9-4762-9113-f8c2ebd538e4","Type":"ContainerStarted","Data":"d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d"}
Jul 28 16:23:27.556411 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:27.555997    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc" event={"ID":"ee779df5-c5a9-4762-9113-f8c2ebd538e4","Type":"ContainerStarted","Data":"3931286eb1d09e0cc05cae986a46464505e62b4515b6b7863bcdbed114eb5896"}
Jul 28 16:23:27.556411 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:27.556029    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:23:27.579275 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:27.579217    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc" podStartSLOduration=1.579200237 podStartE2EDuration="1.579200237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:23:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:23:27.577647727 +0000 UTC m=+1333.722026279" watchObservedRunningTime="2026-07-28 16:23:27.579200237 +0000 UTC m=+1333.723578852"
Jul 28 16:23:28.528730 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:28.528691    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt" podUID="00f7340e-bc02-4848-8e61-5e66fa3d99f1" containerName="sequence-graph-7c1b1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:23:33.529368 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:33.529325    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt" podUID="00f7340e-bc02-4848-8e61-5e66fa3d99f1" containerName="sequence-graph-7c1b1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:23:33.566465 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:33.566433    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:23:38.528634 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:38.528540    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt" podUID="00f7340e-bc02-4848-8e61-5e66fa3d99f1" containerName="sequence-graph-7c1b1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:23:38.529219 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:38.528678    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:43.529114 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:43.529048    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt" podUID="00f7340e-bc02-4848-8e61-5e66fa3d99f1" containerName="sequence-graph-7c1b1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:23:48.528159 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:48.528115    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt" podUID="00f7340e-bc02-4848-8e61-5e66fa3d99f1" containerName="sequence-graph-7c1b1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:23:53.528365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:53.528323    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt" podUID="00f7340e-bc02-4848-8e61-5e66fa3d99f1" containerName="sequence-graph-7c1b1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:23:56.347111 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.347075    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:56.437820 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.437784    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00f7340e-bc02-4848-8e61-5e66fa3d99f1-proxy-tls\") pod \"00f7340e-bc02-4848-8e61-5e66fa3d99f1\" (UID: \"00f7340e-bc02-4848-8e61-5e66fa3d99f1\") "
Jul 28 16:23:56.437994 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.437883    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f7340e-bc02-4848-8e61-5e66fa3d99f1-openshift-service-ca-bundle\") pod \"00f7340e-bc02-4848-8e61-5e66fa3d99f1\" (UID: \"00f7340e-bc02-4848-8e61-5e66fa3d99f1\") "
Jul 28 16:23:56.438277 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.438248    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00f7340e-bc02-4848-8e61-5e66fa3d99f1-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "00f7340e-bc02-4848-8e61-5e66fa3d99f1" (UID: "00f7340e-bc02-4848-8e61-5e66fa3d99f1"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:23:56.439915 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.439884    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f7340e-bc02-4848-8e61-5e66fa3d99f1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "00f7340e-bc02-4848-8e61-5e66fa3d99f1" (UID: "00f7340e-bc02-4848-8e61-5e66fa3d99f1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:23:56.538475 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.538434    2575 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00f7340e-bc02-4848-8e61-5e66fa3d99f1-proxy-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:23:56.538475 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.538469    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f7340e-bc02-4848-8e61-5e66fa3d99f1-openshift-service-ca-bundle\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:23:56.662585 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.662541    2575 generic.go:356] "Generic (PLEG): container finished" podID="00f7340e-bc02-4848-8e61-5e66fa3d99f1" containerID="af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b" exitCode=0
Jul 28 16:23:56.662763 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.662591    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"
Jul 28 16:23:56.662763 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.662592    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt" event={"ID":"00f7340e-bc02-4848-8e61-5e66fa3d99f1","Type":"ContainerDied","Data":"af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b"}
Jul 28 16:23:56.662763 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.662705    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt" event={"ID":"00f7340e-bc02-4848-8e61-5e66fa3d99f1","Type":"ContainerDied","Data":"ed11a4c0ee1bc4f28619ede73029b7f9c5bbaad4723c29966a3723f673bb9ea0"}
Jul 28 16:23:56.662763 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.662734    2575 scope.go:117] "RemoveContainer" containerID="af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b"
Jul 28 16:23:56.670909 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.670890    2575 scope.go:117] "RemoveContainer" containerID="af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b"
Jul 28 16:23:56.671244 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:23:56.671224    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b\": container with ID starting with af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b not found: ID does not exist" containerID="af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b"
Jul 28 16:23:56.671293 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.671255    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b"} err="failed to get container status \"af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b\": rpc error: code = NotFound desc = could not find container \"af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b\": container with ID starting with af4d28a1c1c5cdf8cca05656a0814ee09ce822b86409bea1bd4e47754d8a036b not found: ID does not exist"
Jul 28 16:23:56.681669 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.681637    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"]
Jul 28 16:23:56.687204 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:56.687178    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7c1b1-5f49957547-pnznt"]
Jul 28 16:23:58.444362 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:23:58.444323    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00f7340e-bc02-4848-8e61-5e66fa3d99f1" path="/var/lib/kubelet/pods/00f7340e-bc02-4848-8e61-5e66fa3d99f1/volumes"
Jul 28 16:24:26.502839 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.502801    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"]
Jul 28 16:24:26.503301 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.503078    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f7340e-bc02-4848-8e61-5e66fa3d99f1" containerName="sequence-graph-7c1b1"
Jul 28 16:24:26.503301 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.503090    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f7340e-bc02-4848-8e61-5e66fa3d99f1" containerName="sequence-graph-7c1b1"
Jul 28 16:24:26.503301 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.503156    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f7340e-bc02-4848-8e61-5e66fa3d99f1" containerName="sequence-graph-7c1b1"
Jul 28 16:24:26.507178 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.507157    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:24:26.509566 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.509535    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-c3dc2-serving-cert\""
Jul 28 16:24:26.509718 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.509592    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-c3dc2-kube-rbac-proxy-sar-config\""
Jul 28 16:24:26.516014 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.515982    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"]
Jul 28 16:24:26.583648 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.583612    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/365cb4ed-4f70-41dd-8170-92d3d598570c-proxy-tls\") pod \"sequence-graph-c3dc2-7bdd665dff-5qxg6\" (UID: \"365cb4ed-4f70-41dd-8170-92d3d598570c\") " pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:24:26.583845 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.583658    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/365cb4ed-4f70-41dd-8170-92d3d598570c-openshift-service-ca-bundle\") pod \"sequence-graph-c3dc2-7bdd665dff-5qxg6\" (UID: \"365cb4ed-4f70-41dd-8170-92d3d598570c\") " pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:24:26.684860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.684824    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/365cb4ed-4f70-41dd-8170-92d3d598570c-proxy-tls\") pod \"sequence-graph-c3dc2-7bdd665dff-5qxg6\" (UID: \"365cb4ed-4f70-41dd-8170-92d3d598570c\") " pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:24:26.685029 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.684869    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/365cb4ed-4f70-41dd-8170-92d3d598570c-openshift-service-ca-bundle\") pod \"sequence-graph-c3dc2-7bdd665dff-5qxg6\" (UID: \"365cb4ed-4f70-41dd-8170-92d3d598570c\") " pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:24:26.685572 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.685552    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/365cb4ed-4f70-41dd-8170-92d3d598570c-openshift-service-ca-bundle\") pod \"sequence-graph-c3dc2-7bdd665dff-5qxg6\" (UID: \"365cb4ed-4f70-41dd-8170-92d3d598570c\") " pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:24:26.687365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.687342    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/365cb4ed-4f70-41dd-8170-92d3d598570c-proxy-tls\") pod \"sequence-graph-c3dc2-7bdd665dff-5qxg6\" (UID: \"365cb4ed-4f70-41dd-8170-92d3d598570c\") " pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:24:26.818181 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.818048    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:24:26.949562 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:26.949526    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"]
Jul 28 16:24:26.952766 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:24:26.952731    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod365cb4ed_4f70_41dd_8170_92d3d598570c.slice/crio-21f7078d7fada8084a49080cfc62f8c8bd8e93161614712537ca13b34354a20f WatchSource:0}: Error finding container 21f7078d7fada8084a49080cfc62f8c8bd8e93161614712537ca13b34354a20f: Status 404 returned error can't find the container with id 21f7078d7fada8084a49080cfc62f8c8bd8e93161614712537ca13b34354a20f
Jul 28 16:24:27.783736 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:27.783692    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6" event={"ID":"365cb4ed-4f70-41dd-8170-92d3d598570c","Type":"ContainerStarted","Data":"a3b89333f46e789ab60298bf36df648943e2ffbaf9e94aae715ce35d8faff083"}
Jul 28 16:24:27.784154 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:27.783790    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6" event={"ID":"365cb4ed-4f70-41dd-8170-92d3d598570c","Type":"ContainerStarted","Data":"21f7078d7fada8084a49080cfc62f8c8bd8e93161614712537ca13b34354a20f"}
Jul 28 16:24:27.784154 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:27.783822    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:24:27.801819 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:27.801751    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6" podStartSLOduration=1.801735327 podStartE2EDuration="1.801735327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:24:27.800814907 +0000 UTC m=+1393.945193459" watchObservedRunningTime="2026-07-28 16:24:27.801735327 +0000 UTC m=+1393.946113880"
Jul 28 16:24:33.792970 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:24:33.792931    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:31:41.115653 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:31:41.112467    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"]
Jul 28 16:31:41.115653 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:31:41.112717    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc" podUID="ee779df5-c5a9-4762-9113-f8c2ebd538e4" containerName="ensemble-graph-e5aa9" containerID="cri-o://d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d" gracePeriod=30
Jul 28 16:31:43.564812 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:31:43.564769    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc" podUID="ee779df5-c5a9-4762-9113-f8c2ebd538e4" containerName="ensemble-graph-e5aa9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:31:48.564033 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:31:48.563984    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc" podUID="ee779df5-c5a9-4762-9113-f8c2ebd538e4" containerName="ensemble-graph-e5aa9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:31:53.564823 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:31:53.564785    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc" podUID="ee779df5-c5a9-4762-9113-f8c2ebd538e4" containerName="ensemble-graph-e5aa9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:31:53.565244 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:31:53.564898    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:31:58.564493 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:31:58.564454    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc" podUID="ee779df5-c5a9-4762-9113-f8c2ebd538e4" containerName="ensemble-graph-e5aa9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:32:03.564838 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:03.564793    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc" podUID="ee779df5-c5a9-4762-9113-f8c2ebd538e4" containerName="ensemble-graph-e5aa9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:32:08.564038 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:08.563997    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc" podUID="ee779df5-c5a9-4762-9113-f8c2ebd538e4" containerName="ensemble-graph-e5aa9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:32:11.267438 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.267408    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:32:11.365901 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.365862    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee779df5-c5a9-4762-9113-f8c2ebd538e4-openshift-service-ca-bundle\") pod \"ee779df5-c5a9-4762-9113-f8c2ebd538e4\" (UID: \"ee779df5-c5a9-4762-9113-f8c2ebd538e4\") "
Jul 28 16:32:11.366128 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.365964    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee779df5-c5a9-4762-9113-f8c2ebd538e4-proxy-tls\") pod \"ee779df5-c5a9-4762-9113-f8c2ebd538e4\" (UID: \"ee779df5-c5a9-4762-9113-f8c2ebd538e4\") "
Jul 28 16:32:11.366296 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.366272    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee779df5-c5a9-4762-9113-f8c2ebd538e4-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "ee779df5-c5a9-4762-9113-f8c2ebd538e4" (UID: "ee779df5-c5a9-4762-9113-f8c2ebd538e4"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:32:11.368058 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.367994    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee779df5-c5a9-4762-9113-f8c2ebd538e4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ee779df5-c5a9-4762-9113-f8c2ebd538e4" (UID: "ee779df5-c5a9-4762-9113-f8c2ebd538e4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:32:11.467485 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.467447    2575 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee779df5-c5a9-4762-9113-f8c2ebd538e4-proxy-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:32:11.467485 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.467474    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee779df5-c5a9-4762-9113-f8c2ebd538e4-openshift-service-ca-bundle\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:32:11.498976 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.498932    2575 generic.go:356] "Generic (PLEG): container finished" podID="ee779df5-c5a9-4762-9113-f8c2ebd538e4" containerID="d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d" exitCode=0
Jul 28 16:32:11.498976 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.498977    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"
Jul 28 16:32:11.499204 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.499001    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc" event={"ID":"ee779df5-c5a9-4762-9113-f8c2ebd538e4","Type":"ContainerDied","Data":"d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d"}
Jul 28 16:32:11.499204 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.499055    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc" event={"ID":"ee779df5-c5a9-4762-9113-f8c2ebd538e4","Type":"ContainerDied","Data":"3931286eb1d09e0cc05cae986a46464505e62b4515b6b7863bcdbed114eb5896"}
Jul 28 16:32:11.499204 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.499085    2575 scope.go:117] "RemoveContainer" containerID="d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d"
Jul 28 16:32:11.507178 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.507159    2575 scope.go:117] "RemoveContainer" containerID="d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d"
Jul 28 16:32:11.507428 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:32:11.507412    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d\": container with ID starting with d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d not found: ID does not exist" containerID="d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d"
Jul 28 16:32:11.507483 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.507437    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d"} err="failed to get container status \"d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d\": rpc error: code = NotFound desc = could not find container \"d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d\": container with ID starting with d8a5fc6b402da43bd05871125b322d8ba4f85d464af0cdbd874f3cda6a52df9d not found: ID does not exist"
Jul 28 16:32:11.521960 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.521923    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"]
Jul 28 16:32:11.525700 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:11.525673    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-e5aa9-657946d8cb-xvlqc"]
Jul 28 16:32:12.443894 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:12.443859    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee779df5-c5a9-4762-9113-f8c2ebd538e4" path="/var/lib/kubelet/pods/ee779df5-c5a9-4762-9113-f8c2ebd538e4/volumes"
Jul 28 16:32:41.195340 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.195254    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"]
Jul 28 16:32:41.195909 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.195522    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6" podUID="365cb4ed-4f70-41dd-8170-92d3d598570c" containerName="sequence-graph-c3dc2" containerID="cri-o://a3b89333f46e789ab60298bf36df648943e2ffbaf9e94aae715ce35d8faff083" gracePeriod=30
Jul 28 16:32:41.390509 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.390473    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"]
Jul 28 16:32:41.390787 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.390774    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee779df5-c5a9-4762-9113-f8c2ebd538e4" containerName="ensemble-graph-e5aa9"
Jul 28 16:32:41.390833 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.390788    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee779df5-c5a9-4762-9113-f8c2ebd538e4" containerName="ensemble-graph-e5aa9"
Jul 28 16:32:41.390833 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.390827    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee779df5-c5a9-4762-9113-f8c2ebd538e4" containerName="ensemble-graph-e5aa9"
Jul 28 16:32:41.393813 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.393795    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:32:41.395971 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.395944    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-a9ea4-kube-rbac-proxy-sar-config\""
Jul 28 16:32:41.396113 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.396069    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-a9ea4-serving-cert\""
Jul 28 16:32:41.404503 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.404475    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"]
Jul 28 16:32:41.517365 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.517322    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-openshift-service-ca-bundle\") pod \"splitter-graph-a9ea4-7dbf9cdd7d-smspf\" (UID: \"80ce2218-d7c9-40de-a294-d7db4fbbb4bc\") " pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:32:41.517559 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.517401    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-proxy-tls\") pod \"splitter-graph-a9ea4-7dbf9cdd7d-smspf\" (UID: \"80ce2218-d7c9-40de-a294-d7db4fbbb4bc\") " pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:32:41.618165 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.618120    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-openshift-service-ca-bundle\") pod \"splitter-graph-a9ea4-7dbf9cdd7d-smspf\" (UID: \"80ce2218-d7c9-40de-a294-d7db4fbbb4bc\") " pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:32:41.618359 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.618200    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-proxy-tls\") pod \"splitter-graph-a9ea4-7dbf9cdd7d-smspf\" (UID: \"80ce2218-d7c9-40de-a294-d7db4fbbb4bc\") " pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:32:41.618359 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:32:41.618304    2575 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-a9ea4-serving-cert: secret "splitter-graph-a9ea4-serving-cert" not found
Jul 28 16:32:41.618441 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:32:41.618370    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-proxy-tls podName:80ce2218-d7c9-40de-a294-d7db4fbbb4bc nodeName:}" failed. No retries permitted until 2026-07-28 16:32:42.118352503 +0000 UTC m=+1888.262731033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-proxy-tls") pod "splitter-graph-a9ea4-7dbf9cdd7d-smspf" (UID: "80ce2218-d7c9-40de-a294-d7db4fbbb4bc") : secret "splitter-graph-a9ea4-serving-cert" not found
Jul 28 16:32:41.618839 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:41.618820    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-openshift-service-ca-bundle\") pod \"splitter-graph-a9ea4-7dbf9cdd7d-smspf\" (UID: \"80ce2218-d7c9-40de-a294-d7db4fbbb4bc\") " pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:32:42.122850 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:42.122805    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-proxy-tls\") pod \"splitter-graph-a9ea4-7dbf9cdd7d-smspf\" (UID: \"80ce2218-d7c9-40de-a294-d7db4fbbb4bc\") " pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:32:42.125287 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:42.125265    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-proxy-tls\") pod \"splitter-graph-a9ea4-7dbf9cdd7d-smspf\" (UID: \"80ce2218-d7c9-40de-a294-d7db4fbbb4bc\") " pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:32:42.305156 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:42.305116    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:32:42.445694 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:42.445661    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"]
Jul 28 16:32:42.449356 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:32:42.449327    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80ce2218_d7c9_40de_a294_d7db4fbbb4bc.slice/crio-4833551e02c31e2177421a37988ad6be9b2b462d7b5ddef25fc212fa0f6a4f6e WatchSource:0}: Error finding container 4833551e02c31e2177421a37988ad6be9b2b462d7b5ddef25fc212fa0f6a4f6e: Status 404 returned error can't find the container with id 4833551e02c31e2177421a37988ad6be9b2b462d7b5ddef25fc212fa0f6a4f6e
Jul 28 16:32:42.612544 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:42.612509    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf" event={"ID":"80ce2218-d7c9-40de-a294-d7db4fbbb4bc","Type":"ContainerStarted","Data":"1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832"}
Jul 28 16:32:42.612728 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:42.612565    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf" event={"ID":"80ce2218-d7c9-40de-a294-d7db4fbbb4bc","Type":"ContainerStarted","Data":"4833551e02c31e2177421a37988ad6be9b2b462d7b5ddef25fc212fa0f6a4f6e"}
Jul 28 16:32:42.612728 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:42.612609    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:32:43.791016 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:43.790977    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6" podUID="365cb4ed-4f70-41dd-8170-92d3d598570c" containerName="sequence-graph-c3dc2" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:32:48.621956 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:48.621927    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:32:48.640379 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:48.640309    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf" podStartSLOduration=7.640286397 podStartE2EDuration="7.640286397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:32:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:32:42.644562062 +0000 UTC m=+1888.788940614" watchObservedRunningTime="2026-07-28 16:32:48.640286397 +0000 UTC m=+1894.784664950"
Jul 28 16:32:48.791538 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:48.791493    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6" podUID="365cb4ed-4f70-41dd-8170-92d3d598570c" containerName="sequence-graph-c3dc2" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:32:51.440379 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:51.440279    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"]
Jul 28 16:32:51.440832 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:51.440504    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf" podUID="80ce2218-d7c9-40de-a294-d7db4fbbb4bc" containerName="splitter-graph-a9ea4" containerID="cri-o://1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832" gracePeriod=30
Jul 28 16:32:53.621048 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:53.621002    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf" podUID="80ce2218-d7c9-40de-a294-d7db4fbbb4bc" containerName="splitter-graph-a9ea4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:32:53.791649 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:53.791603    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6" podUID="365cb4ed-4f70-41dd-8170-92d3d598570c" containerName="sequence-graph-c3dc2" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:32:53.791844 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:53.791726    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:32:58.620978 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:58.620935    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf" podUID="80ce2218-d7c9-40de-a294-d7db4fbbb4bc" containerName="splitter-graph-a9ea4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:32:58.791181 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:32:58.791135    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6" podUID="365cb4ed-4f70-41dd-8170-92d3d598570c" containerName="sequence-graph-c3dc2" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:33:03.620045 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:03.620005    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf" podUID="80ce2218-d7c9-40de-a294-d7db4fbbb4bc" containerName="splitter-graph-a9ea4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:33:03.620588 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:03.620136    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:33:03.791290 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:03.791245    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6" podUID="365cb4ed-4f70-41dd-8170-92d3d598570c" containerName="sequence-graph-c3dc2" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:33:08.620849 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:08.620809    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf" podUID="80ce2218-d7c9-40de-a294-d7db4fbbb4bc" containerName="splitter-graph-a9ea4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:33:08.791438 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:08.791393    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6" podUID="365cb4ed-4f70-41dd-8170-92d3d598570c" containerName="sequence-graph-c3dc2" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:33:11.724139 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:11.724085    2575 generic.go:356] "Generic (PLEG): container finished" podID="365cb4ed-4f70-41dd-8170-92d3d598570c" containerID="a3b89333f46e789ab60298bf36df648943e2ffbaf9e94aae715ce35d8faff083" exitCode=0
Jul 28 16:33:11.724139 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:11.724138    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6" event={"ID":"365cb4ed-4f70-41dd-8170-92d3d598570c","Type":"ContainerDied","Data":"a3b89333f46e789ab60298bf36df648943e2ffbaf9e94aae715ce35d8faff083"}
Jul 28 16:33:11.843828 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:11.843799    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:33:11.862913 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:11.862882    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/365cb4ed-4f70-41dd-8170-92d3d598570c-proxy-tls\") pod \"365cb4ed-4f70-41dd-8170-92d3d598570c\" (UID: \"365cb4ed-4f70-41dd-8170-92d3d598570c\") "
Jul 28 16:33:11.862913 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:11.862920    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/365cb4ed-4f70-41dd-8170-92d3d598570c-openshift-service-ca-bundle\") pod \"365cb4ed-4f70-41dd-8170-92d3d598570c\" (UID: \"365cb4ed-4f70-41dd-8170-92d3d598570c\") "
Jul 28 16:33:11.863347 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:11.863321    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/365cb4ed-4f70-41dd-8170-92d3d598570c-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "365cb4ed-4f70-41dd-8170-92d3d598570c" (UID: "365cb4ed-4f70-41dd-8170-92d3d598570c"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:33:11.865218 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:11.865189    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/365cb4ed-4f70-41dd-8170-92d3d598570c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "365cb4ed-4f70-41dd-8170-92d3d598570c" (UID: "365cb4ed-4f70-41dd-8170-92d3d598570c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:33:11.963676 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:11.963642    2575 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/365cb4ed-4f70-41dd-8170-92d3d598570c-proxy-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:33:11.963676 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:11.963676    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/365cb4ed-4f70-41dd-8170-92d3d598570c-openshift-service-ca-bundle\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:33:12.729018 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:12.728976    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"
Jul 28 16:33:12.729018 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:12.729009    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6" event={"ID":"365cb4ed-4f70-41dd-8170-92d3d598570c","Type":"ContainerDied","Data":"21f7078d7fada8084a49080cfc62f8c8bd8e93161614712537ca13b34354a20f"}
Jul 28 16:33:12.729492 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:12.729075    2575 scope.go:117] "RemoveContainer" containerID="a3b89333f46e789ab60298bf36df648943e2ffbaf9e94aae715ce35d8faff083"
Jul 28 16:33:12.744338 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:12.744303    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"]
Jul 28 16:33:12.748478 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:12.748433    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3dc2-7bdd665dff-5qxg6"]
Jul 28 16:33:13.621002 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:13.620956    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf" podUID="80ce2218-d7c9-40de-a294-d7db4fbbb4bc" containerName="splitter-graph-a9ea4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:33:14.444044 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:14.444005    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="365cb4ed-4f70-41dd-8170-92d3d598570c" path="/var/lib/kubelet/pods/365cb4ed-4f70-41dd-8170-92d3d598570c/volumes"
Jul 28 16:33:18.620644 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:18.620601    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf" podUID="80ce2218-d7c9-40de-a294-d7db4fbbb4bc" containerName="splitter-graph-a9ea4" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:33:21.575134 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.575083    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:33:21.637162 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.637123    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-proxy-tls\") pod \"80ce2218-d7c9-40de-a294-d7db4fbbb4bc\" (UID: \"80ce2218-d7c9-40de-a294-d7db4fbbb4bc\") "
Jul 28 16:33:21.637341 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.637184    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-openshift-service-ca-bundle\") pod \"80ce2218-d7c9-40de-a294-d7db4fbbb4bc\" (UID: \"80ce2218-d7c9-40de-a294-d7db4fbbb4bc\") "
Jul 28 16:33:21.637556 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.637531    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "80ce2218-d7c9-40de-a294-d7db4fbbb4bc" (UID: "80ce2218-d7c9-40de-a294-d7db4fbbb4bc"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:33:21.639257 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.639232    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "80ce2218-d7c9-40de-a294-d7db4fbbb4bc" (UID: "80ce2218-d7c9-40de-a294-d7db4fbbb4bc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:33:21.737868 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.737832    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-openshift-service-ca-bundle\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:33:21.737868 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.737865    2575 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80ce2218-d7c9-40de-a294-d7db4fbbb4bc-proxy-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:33:21.761802 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.761770    2575 generic.go:356] "Generic (PLEG): container finished" podID="80ce2218-d7c9-40de-a294-d7db4fbbb4bc" containerID="1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832" exitCode=0
Jul 28 16:33:21.761974 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.761818    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"
Jul 28 16:33:21.761974 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.761858    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf" event={"ID":"80ce2218-d7c9-40de-a294-d7db4fbbb4bc","Type":"ContainerDied","Data":"1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832"}
Jul 28 16:33:21.761974 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.761910    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf" event={"ID":"80ce2218-d7c9-40de-a294-d7db4fbbb4bc","Type":"ContainerDied","Data":"4833551e02c31e2177421a37988ad6be9b2b462d7b5ddef25fc212fa0f6a4f6e"}
Jul 28 16:33:21.761974 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.761933    2575 scope.go:117] "RemoveContainer" containerID="1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832"
Jul 28 16:33:21.770196 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.770175    2575 scope.go:117] "RemoveContainer" containerID="1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832"
Jul 28 16:33:21.770493 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:33:21.770473    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832\": container with ID starting with 1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832 not found: ID does not exist" containerID="1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832"
Jul 28 16:33:21.770543 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.770504    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832"} err="failed to get container status \"1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832\": rpc error: code = NotFound desc = could not find container \"1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832\": container with ID starting with 1e579c1e00d440a84475655dd046e99aa9c4166516d185d1458f40151c77e832 not found: ID does not exist"
Jul 28 16:33:21.781956 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.781922    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"]
Jul 28 16:33:21.783929 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:21.783903    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-a9ea4-7dbf9cdd7d-smspf"]
Jul 28 16:33:22.443438 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:22.443401    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80ce2218-d7c9-40de-a294-d7db4fbbb4bc" path="/var/lib/kubelet/pods/80ce2218-d7c9-40de-a294-d7db4fbbb4bc/volumes"
Jul 28 16:33:41.532819 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.532770    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"]
Jul 28 16:33:41.533357 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.533162    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80ce2218-d7c9-40de-a294-d7db4fbbb4bc" containerName="splitter-graph-a9ea4"
Jul 28 16:33:41.533357 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.533181    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="80ce2218-d7c9-40de-a294-d7db4fbbb4bc" containerName="splitter-graph-a9ea4"
Jul 28 16:33:41.533357 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.533203    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="365cb4ed-4f70-41dd-8170-92d3d598570c" containerName="sequence-graph-c3dc2"
Jul 28 16:33:41.533357 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.533209    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="365cb4ed-4f70-41dd-8170-92d3d598570c" containerName="sequence-graph-c3dc2"
Jul 28 16:33:41.533357 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.533252    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="80ce2218-d7c9-40de-a294-d7db4fbbb4bc" containerName="splitter-graph-a9ea4"
Jul 28 16:33:41.533357 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.533262    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="365cb4ed-4f70-41dd-8170-92d3d598570c" containerName="sequence-graph-c3dc2"
Jul 28 16:33:41.537541 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.537518    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:33:41.539999 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.539977    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-k6dfd\""
Jul 28 16:33:41.540296 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.540275    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-d3a3d-kube-rbac-proxy-sar-config\""
Jul 28 16:33:41.540351 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.540317    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-d3a3d-serving-cert\""
Jul 28 16:33:41.540659 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.540644    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 28 16:33:41.545932 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.545905    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"]
Jul 28 16:33:41.599625 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.599587    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a0de90-5b46-4266-a9db-34fe621656e9-openshift-service-ca-bundle\") pod \"switch-graph-d3a3d-6fc78c5b9-pzf8m\" (UID: \"64a0de90-5b46-4266-a9db-34fe621656e9\") " pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:33:41.599796 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.599638    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64a0de90-5b46-4266-a9db-34fe621656e9-proxy-tls\") pod \"switch-graph-d3a3d-6fc78c5b9-pzf8m\" (UID: \"64a0de90-5b46-4266-a9db-34fe621656e9\") " pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:33:41.700819 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.700779    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64a0de90-5b46-4266-a9db-34fe621656e9-proxy-tls\") pod \"switch-graph-d3a3d-6fc78c5b9-pzf8m\" (UID: \"64a0de90-5b46-4266-a9db-34fe621656e9\") " pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:33:41.701028 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.700855    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a0de90-5b46-4266-a9db-34fe621656e9-openshift-service-ca-bundle\") pod \"switch-graph-d3a3d-6fc78c5b9-pzf8m\" (UID: \"64a0de90-5b46-4266-a9db-34fe621656e9\") " pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:33:41.701544 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.701521    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a0de90-5b46-4266-a9db-34fe621656e9-openshift-service-ca-bundle\") pod \"switch-graph-d3a3d-6fc78c5b9-pzf8m\" (UID: \"64a0de90-5b46-4266-a9db-34fe621656e9\") " pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:33:41.703273 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.703253    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64a0de90-5b46-4266-a9db-34fe621656e9-proxy-tls\") pod \"switch-graph-d3a3d-6fc78c5b9-pzf8m\" (UID: \"64a0de90-5b46-4266-a9db-34fe621656e9\") " pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:33:41.848062 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.847966    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:33:41.990626 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:41.990588    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"]
Jul 28 16:33:41.993551 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:33:41.993524    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a0de90_5b46_4266_a9db_34fe621656e9.slice/crio-6df1888e41009065865170d759d677eaa879c1b2cddab0e73c8694834df4df9e WatchSource:0}: Error finding container 6df1888e41009065865170d759d677eaa879c1b2cddab0e73c8694834df4df9e: Status 404 returned error can't find the container with id 6df1888e41009065865170d759d677eaa879c1b2cddab0e73c8694834df4df9e
Jul 28 16:33:42.839059 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:42.839018    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m" event={"ID":"64a0de90-5b46-4266-a9db-34fe621656e9","Type":"ContainerStarted","Data":"603641e6cb9a5d227edbf96568701c0b0a58297a233cec152e5e99e6c0c0cea4"}
Jul 28 16:33:42.839473 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:42.839080    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m" event={"ID":"64a0de90-5b46-4266-a9db-34fe621656e9","Type":"ContainerStarted","Data":"6df1888e41009065865170d759d677eaa879c1b2cddab0e73c8694834df4df9e"}
Jul 28 16:33:42.839473 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:42.839128    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:33:42.855690 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:42.855637    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m" podStartSLOduration=1.8556192409999999 podStartE2EDuration="1.855619241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:33:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:33:42.853920905 +0000 UTC m=+1948.998299459" watchObservedRunningTime="2026-07-28 16:33:42.855619241 +0000 UTC m=+1948.999997793"
Jul 28 16:33:48.848219 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:33:48.848180    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:34:01.699127 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:01.699077    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"]
Jul 28 16:34:01.702491 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:01.702470    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:34:01.704384 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:01.704356    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-49c25-serving-cert\""
Jul 28 16:34:01.704515 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:01.704430    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-49c25-kube-rbac-proxy-sar-config\""
Jul 28 16:34:01.711860 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:01.711825    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"]
Jul 28 16:34:01.865495 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:01.865452    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6722563b-adb3-47fb-bb40-41e7567baf9a-openshift-service-ca-bundle\") pod \"splitter-graph-49c25-56ffcb7444-z2gf6\" (UID: \"6722563b-adb3-47fb-bb40-41e7567baf9a\") " pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:34:01.865699 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:01.865592    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6722563b-adb3-47fb-bb40-41e7567baf9a-proxy-tls\") pod \"splitter-graph-49c25-56ffcb7444-z2gf6\" (UID: \"6722563b-adb3-47fb-bb40-41e7567baf9a\") " pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:34:01.966671 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:01.966626    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6722563b-adb3-47fb-bb40-41e7567baf9a-openshift-service-ca-bundle\") pod \"splitter-graph-49c25-56ffcb7444-z2gf6\" (UID: \"6722563b-adb3-47fb-bb40-41e7567baf9a\") " pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:34:01.966880 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:01.966718    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6722563b-adb3-47fb-bb40-41e7567baf9a-proxy-tls\") pod \"splitter-graph-49c25-56ffcb7444-z2gf6\" (UID: \"6722563b-adb3-47fb-bb40-41e7567baf9a\") " pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:34:01.967401 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:01.967377    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6722563b-adb3-47fb-bb40-41e7567baf9a-openshift-service-ca-bundle\") pod \"splitter-graph-49c25-56ffcb7444-z2gf6\" (UID: \"6722563b-adb3-47fb-bb40-41e7567baf9a\") " pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:34:01.969325 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:01.969303    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6722563b-adb3-47fb-bb40-41e7567baf9a-proxy-tls\") pod \"splitter-graph-49c25-56ffcb7444-z2gf6\" (UID: \"6722563b-adb3-47fb-bb40-41e7567baf9a\") " pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:34:02.013339 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:02.013296    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:34:02.147609 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:02.147572    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"]
Jul 28 16:34:02.150792 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:34:02.150766    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6722563b_adb3_47fb_bb40_41e7567baf9a.slice/crio-082ce5ea2e5231aa5830c2f65d41db3749c1f1090105862eed49fedf0fe61a8a WatchSource:0}: Error finding container 082ce5ea2e5231aa5830c2f65d41db3749c1f1090105862eed49fedf0fe61a8a: Status 404 returned error can't find the container with id 082ce5ea2e5231aa5830c2f65d41db3749c1f1090105862eed49fedf0fe61a8a
Jul 28 16:34:02.914581 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:02.914546    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6" event={"ID":"6722563b-adb3-47fb-bb40-41e7567baf9a","Type":"ContainerStarted","Data":"a9363245a4f3c73af23e63747fffc3bda748a07edbd42747b7339fa08056d72f"}
Jul 28 16:34:02.914999 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:02.914599    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6" event={"ID":"6722563b-adb3-47fb-bb40-41e7567baf9a","Type":"ContainerStarted","Data":"082ce5ea2e5231aa5830c2f65d41db3749c1f1090105862eed49fedf0fe61a8a"}
Jul 28 16:34:02.914999 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:02.914666    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:34:02.931495 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:02.931436    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6" podStartSLOduration=1.931418332 podStartE2EDuration="1.931418332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:34:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:34:02.929792357 +0000 UTC m=+1969.074170910" watchObservedRunningTime="2026-07-28 16:34:02.931418332 +0000 UTC m=+1969.075796884"
Jul 28 16:34:08.923719 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:34:08.923643    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:42:16.371026 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:16.370986    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"]
Jul 28 16:42:16.373197 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:16.371222    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6" podUID="6722563b-adb3-47fb-bb40-41e7567baf9a" containerName="splitter-graph-49c25" containerID="cri-o://a9363245a4f3c73af23e63747fffc3bda748a07edbd42747b7339fa08056d72f" gracePeriod=30
Jul 28 16:42:18.922980 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:18.922938    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6" podUID="6722563b-adb3-47fb-bb40-41e7567baf9a" containerName="splitter-graph-49c25" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:42:23.927256 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:23.927205    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6" podUID="6722563b-adb3-47fb-bb40-41e7567baf9a" containerName="splitter-graph-49c25" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:42:28.922602 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:28.922561    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6" podUID="6722563b-adb3-47fb-bb40-41e7567baf9a" containerName="splitter-graph-49c25" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:42:28.923126 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:28.922715    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:42:33.922932 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:33.922893    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6" podUID="6722563b-adb3-47fb-bb40-41e7567baf9a" containerName="splitter-graph-49c25" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:42:38.922345 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:38.922308    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6" podUID="6722563b-adb3-47fb-bb40-41e7567baf9a" containerName="splitter-graph-49c25" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:42:43.922703 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:43.922651    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6" podUID="6722563b-adb3-47fb-bb40-41e7567baf9a" containerName="splitter-graph-49c25" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:42:46.832800 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:46.832766    2575 generic.go:356] "Generic (PLEG): container finished" podID="6722563b-adb3-47fb-bb40-41e7567baf9a" containerID="a9363245a4f3c73af23e63747fffc3bda748a07edbd42747b7339fa08056d72f" exitCode=0
Jul 28 16:42:46.833210 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:46.832852    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6" event={"ID":"6722563b-adb3-47fb-bb40-41e7567baf9a","Type":"ContainerDied","Data":"a9363245a4f3c73af23e63747fffc3bda748a07edbd42747b7339fa08056d72f"}
Jul 28 16:42:47.010572 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:47.010542    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:42:47.092428 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:47.092327    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6722563b-adb3-47fb-bb40-41e7567baf9a-proxy-tls\") pod \"6722563b-adb3-47fb-bb40-41e7567baf9a\" (UID: \"6722563b-adb3-47fb-bb40-41e7567baf9a\") "
Jul 28 16:42:47.092614 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:47.092453    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6722563b-adb3-47fb-bb40-41e7567baf9a-openshift-service-ca-bundle\") pod \"6722563b-adb3-47fb-bb40-41e7567baf9a\" (UID: \"6722563b-adb3-47fb-bb40-41e7567baf9a\") "
Jul 28 16:42:47.092828 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:47.092805    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6722563b-adb3-47fb-bb40-41e7567baf9a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "6722563b-adb3-47fb-bb40-41e7567baf9a" (UID: "6722563b-adb3-47fb-bb40-41e7567baf9a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:42:47.094551 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:47.094520    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6722563b-adb3-47fb-bb40-41e7567baf9a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6722563b-adb3-47fb-bb40-41e7567baf9a" (UID: "6722563b-adb3-47fb-bb40-41e7567baf9a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:42:47.193169 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:47.193124    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6722563b-adb3-47fb-bb40-41e7567baf9a-openshift-service-ca-bundle\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:42:47.193169 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:47.193160    2575 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6722563b-adb3-47fb-bb40-41e7567baf9a-proxy-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:42:47.837808 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:47.837776    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"
Jul 28 16:42:47.837808 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:47.837802    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6" event={"ID":"6722563b-adb3-47fb-bb40-41e7567baf9a","Type":"ContainerDied","Data":"082ce5ea2e5231aa5830c2f65d41db3749c1f1090105862eed49fedf0fe61a8a"}
Jul 28 16:42:47.838332 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:47.837857    2575 scope.go:117] "RemoveContainer" containerID="a9363245a4f3c73af23e63747fffc3bda748a07edbd42747b7339fa08056d72f"
Jul 28 16:42:47.858412 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:47.858375    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"]
Jul 28 16:42:47.864884 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:47.864854    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-49c25-56ffcb7444-z2gf6"]
Jul 28 16:42:48.443413 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:42:48.443377    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6722563b-adb3-47fb-bb40-41e7567baf9a" path="/var/lib/kubelet/pods/6722563b-adb3-47fb-bb40-41e7567baf9a/volumes"
Jul 28 16:50:00.684897 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:00.684858    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"]
Jul 28 16:50:00.687473 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:00.685086    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m" podUID="64a0de90-5b46-4266-a9db-34fe621656e9" containerName="switch-graph-d3a3d" containerID="cri-o://603641e6cb9a5d227edbf96568701c0b0a58297a233cec152e5e99e6c0c0cea4" gracePeriod=30
Jul 28 16:50:01.718558 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.718520    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c6qjl/must-gather-h477h"]
Jul 28 16:50:01.718946 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.718805    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6722563b-adb3-47fb-bb40-41e7567baf9a" containerName="splitter-graph-49c25"
Jul 28 16:50:01.718946 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.718817    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6722563b-adb3-47fb-bb40-41e7567baf9a" containerName="splitter-graph-49c25"
Jul 28 16:50:01.718946 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.718871    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6722563b-adb3-47fb-bb40-41e7567baf9a" containerName="splitter-graph-49c25"
Jul 28 16:50:01.721267 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.721251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6qjl/must-gather-h477h"
Jul 28 16:50:01.723726 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.723700    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c6qjl\"/\"default-dockercfg-t7nns\""
Jul 28 16:50:01.724191 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.724174    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c6qjl\"/\"kube-root-ca.crt\""
Jul 28 16:50:01.724288 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.724210    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c6qjl\"/\"openshift-service-ca.crt\""
Jul 28 16:50:01.736287 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.736265    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c6qjl/must-gather-h477h"]
Jul 28 16:50:01.817249 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.817201    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff-must-gather-output\") pod \"must-gather-h477h\" (UID: \"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff\") " pod="openshift-must-gather-c6qjl/must-gather-h477h"
Jul 28 16:50:01.817429 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.817273    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vglvt\" (UniqueName: \"kubernetes.io/projected/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff-kube-api-access-vglvt\") pod \"must-gather-h477h\" (UID: \"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff\") " pod="openshift-must-gather-c6qjl/must-gather-h477h"
Jul 28 16:50:01.918334 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.918292    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff-must-gather-output\") pod \"must-gather-h477h\" (UID: \"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff\") " pod="openshift-must-gather-c6qjl/must-gather-h477h"
Jul 28 16:50:01.918723 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.918356    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vglvt\" (UniqueName: \"kubernetes.io/projected/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff-kube-api-access-vglvt\") pod \"must-gather-h477h\" (UID: \"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff\") " pod="openshift-must-gather-c6qjl/must-gather-h477h"
Jul 28 16:50:01.918723 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.918640    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff-must-gather-output\") pod \"must-gather-h477h\" (UID: \"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff\") " pod="openshift-must-gather-c6qjl/must-gather-h477h"
Jul 28 16:50:01.926464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:01.926432    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vglvt\" (UniqueName: \"kubernetes.io/projected/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff-kube-api-access-vglvt\") pod \"must-gather-h477h\" (UID: \"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff\") " pod="openshift-must-gather-c6qjl/must-gather-h477h"
Jul 28 16:50:02.038030 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:02.037934    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6qjl/must-gather-h477h"
Jul 28 16:50:02.176186 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:02.176153    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c6qjl/must-gather-h477h"]
Jul 28 16:50:02.179748 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:50:02.179706    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3366c9c7_d68b_4a25_b2d6_b12dea4f33ff.slice/crio-c599cd38cb785dacbcd28c686d71e4bb749b86933c51cab4e151de599fc8c4f9 WatchSource:0}: Error finding container c599cd38cb785dacbcd28c686d71e4bb749b86933c51cab4e151de599fc8c4f9: Status 404 returned error can't find the container with id c599cd38cb785dacbcd28c686d71e4bb749b86933c51cab4e151de599fc8c4f9
Jul 28 16:50:02.181927 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:02.181909    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 16:50:02.420937 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:02.420844    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6qjl/must-gather-h477h" event={"ID":"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff","Type":"ContainerStarted","Data":"c599cd38cb785dacbcd28c686d71e4bb749b86933c51cab4e151de599fc8c4f9"}
Jul 28 16:50:03.849656 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:03.849607    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m" podUID="64a0de90-5b46-4266-a9db-34fe621656e9" containerName="switch-graph-d3a3d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:50:08.847029 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:08.846940    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m" podUID="64a0de90-5b46-4266-a9db-34fe621656e9" containerName="switch-graph-d3a3d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:50:09.452422 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:09.452381    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6qjl/must-gather-h477h" event={"ID":"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff","Type":"ContainerStarted","Data":"5b0db28ca6d021a431a408ae655348ce84fec8a509bfff2e005fe68aebc9937f"}
Jul 28 16:50:09.452756 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:09.452723    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6qjl/must-gather-h477h" event={"ID":"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff","Type":"ContainerStarted","Data":"d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d"}
Jul 28 16:50:09.469221 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:09.469169    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c6qjl/must-gather-h477h" podStartSLOduration=2.174878499 podStartE2EDuration="8.469146158s" totalImagesPullingTime="6.294267659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:50:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:50:09.468188938 +0000 UTC m=+2935.612567497" watchObservedRunningTime="2026-07-28 16:50:09.469146158 +0000 UTC m=+2935.613524711"
Jul 28 16:50:13.847825 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:13.847782    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m" podUID="64a0de90-5b46-4266-a9db-34fe621656e9" containerName="switch-graph-d3a3d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:50:13.848396 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:13.847901    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:50:17.936786 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:17.936745    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:18.804920 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:18.804888    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:18.846937 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:18.846899    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m" podUID="64a0de90-5b46-4266-a9db-34fe621656e9" containerName="switch-graph-d3a3d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:50:19.654767 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:19.654739    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:20.474176 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:20.474125    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:21.302367 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:21.302332    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:22.152779 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:22.152750    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:23.031653 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:23.031624    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:23.847671 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:23.847632    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m" podUID="64a0de90-5b46-4266-a9db-34fe621656e9" containerName="switch-graph-d3a3d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:50:23.892637 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:23.892601    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:24.750474 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:24.750444    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:25.581531 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:25.581503    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:26.395164 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:26.395125    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:27.211355 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:27.211320    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:28.043899 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:28.043866    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:28.847333 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:28.847294    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m" podUID="64a0de90-5b46-4266-a9db-34fe621656e9" containerName="switch-graph-d3a3d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:50:28.922314 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:28.922277    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d3a3d-6fc78c5b9-pzf8m_64a0de90-5b46-4266-a9db-34fe621656e9/switch-graph-d3a3d/0.log"
Jul 28 16:50:31.542611 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:31.542571    2575 generic.go:356] "Generic (PLEG): container finished" podID="64a0de90-5b46-4266-a9db-34fe621656e9" containerID="603641e6cb9a5d227edbf96568701c0b0a58297a233cec152e5e99e6c0c0cea4" exitCode=0
Jul 28 16:50:31.543036 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:31.542629    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m" event={"ID":"64a0de90-5b46-4266-a9db-34fe621656e9","Type":"ContainerDied","Data":"603641e6cb9a5d227edbf96568701c0b0a58297a233cec152e5e99e6c0c0cea4"}
Jul 28 16:50:31.929863 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:31.929832    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:50:31.990560 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:31.990523    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64a0de90-5b46-4266-a9db-34fe621656e9-proxy-tls\") pod \"64a0de90-5b46-4266-a9db-34fe621656e9\" (UID: \"64a0de90-5b46-4266-a9db-34fe621656e9\") "
Jul 28 16:50:31.990749 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:31.990635    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a0de90-5b46-4266-a9db-34fe621656e9-openshift-service-ca-bundle\") pod \"64a0de90-5b46-4266-a9db-34fe621656e9\" (UID: \"64a0de90-5b46-4266-a9db-34fe621656e9\") "
Jul 28 16:50:31.991011 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:31.990982    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64a0de90-5b46-4266-a9db-34fe621656e9-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "64a0de90-5b46-4266-a9db-34fe621656e9" (UID: "64a0de90-5b46-4266-a9db-34fe621656e9"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:50:31.992639 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:31.992617    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64a0de90-5b46-4266-a9db-34fe621656e9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "64a0de90-5b46-4266-a9db-34fe621656e9" (UID: "64a0de90-5b46-4266-a9db-34fe621656e9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:50:32.092084 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:32.091998    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a0de90-5b46-4266-a9db-34fe621656e9-openshift-service-ca-bundle\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:50:32.092084 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:32.092030    2575 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64a0de90-5b46-4266-a9db-34fe621656e9-proxy-tls\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:50:32.548232 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:32.548194    2575 generic.go:356] "Generic (PLEG): container finished" podID="3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" containerID="d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d" exitCode=0
Jul 28 16:50:32.548657 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:32.548268    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6qjl/must-gather-h477h" event={"ID":"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff","Type":"ContainerDied","Data":"d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d"}
Jul 28 16:50:32.548728 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:32.548672    2575 scope.go:117] "RemoveContainer" containerID="d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d"
Jul 28 16:50:32.549582 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:32.549556    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m" event={"ID":"64a0de90-5b46-4266-a9db-34fe621656e9","Type":"ContainerDied","Data":"6df1888e41009065865170d759d677eaa879c1b2cddab0e73c8694834df4df9e"}
Jul 28 16:50:32.549703 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:32.549605    2575 scope.go:117] "RemoveContainer" containerID="603641e6cb9a5d227edbf96568701c0b0a58297a233cec152e5e99e6c0c0cea4"
Jul 28 16:50:32.549703 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:32.549564    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"
Jul 28 16:50:32.574755 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:32.574711    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"]
Jul 28 16:50:32.578336 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:32.578307    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d3a3d-6fc78c5b9-pzf8m"]
Jul 28 16:50:32.737260 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:32.737215    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-c6qjl_must-gather-h477h_3366c9c7-d68b-4a25-b2d6-b12dea4f33ff/gather/0.log"
Jul 28 16:50:34.443927 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:34.443893    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64a0de90-5b46-4266-a9db-34fe621656e9" path="/var/lib/kubelet/pods/64a0de90-5b46-4266-a9db-34fe621656e9/volumes"
Jul 28 16:50:35.918321 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:35.918231    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9fplh_d700d5d0-afa0-407f-b47a-768e795a3949/global-pull-secret-syncer/0.log"
Jul 28 16:50:36.151482 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:36.151443    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vrwd6_ff7d1e7e-c218-4b9e-8cfe-b63c9bd54a28/konnectivity-agent/0.log"
Jul 28 16:50:36.204351 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:36.204272    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-224.ec2.internal_7301dfd284c93177b5a9311d5aa95a82/haproxy/0.log"
Jul 28 16:50:38.175120 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.175059    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-c6qjl/must-gather-h477h"]
Jul 28 16:50:38.175607 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.175313    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-c6qjl/must-gather-h477h" podUID="3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" containerName="copy" containerID="cri-o://5b0db28ca6d021a431a408ae655348ce84fec8a509bfff2e005fe68aebc9937f" gracePeriod=2
Jul 28 16:50:38.177648 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:50:38.177612    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-h477h\" is forbidden: User \"system:node:ip-10-0-140-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-c6qjl\": no relationship found between node 'ip-10-0-140-224.ec2.internal' and this object" podUID="3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" pod="openshift-must-gather-c6qjl/must-gather-h477h"
Jul 28 16:50:38.178627 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.178600    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-c6qjl/must-gather-h477h"]
Jul 28 16:50:38.405614 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.405589    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-c6qjl_must-gather-h477h_3366c9c7-d68b-4a25-b2d6-b12dea4f33ff/copy/0.log"
Jul 28 16:50:38.407160 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.407138    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6qjl/must-gather-h477h"
Jul 28 16:50:38.409190 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:50:38.409163    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-h477h\" is forbidden: User \"system:node:ip-10-0-140-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-c6qjl\": no relationship found between node 'ip-10-0-140-224.ec2.internal' and this object" podUID="3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" pod="openshift-must-gather-c6qjl/must-gather-h477h"
Jul 28 16:50:38.545979 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.545937    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vglvt\" (UniqueName: \"kubernetes.io/projected/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff-kube-api-access-vglvt\") pod \"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff\" (UID: \"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff\") "
Jul 28 16:50:38.546181 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.546017    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff-must-gather-output\") pod \"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff\" (UID: \"3366c9c7-d68b-4a25-b2d6-b12dea4f33ff\") "
Jul 28 16:50:38.547626 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.547596    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" (UID: "3366c9c7-d68b-4a25-b2d6-b12dea4f33ff"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 16:50:38.548315 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.548283    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff-kube-api-access-vglvt" (OuterVolumeSpecName: "kube-api-access-vglvt") pod "3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" (UID: "3366c9c7-d68b-4a25-b2d6-b12dea4f33ff"). InnerVolumeSpecName "kube-api-access-vglvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:50:38.574464 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.574436    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-c6qjl_must-gather-h477h_3366c9c7-d68b-4a25-b2d6-b12dea4f33ff/copy/0.log"
Jul 28 16:50:38.574807 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.574779    2575 generic.go:356] "Generic (PLEG): container finished" podID="3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" containerID="5b0db28ca6d021a431a408ae655348ce84fec8a509bfff2e005fe68aebc9937f" exitCode=143
Jul 28 16:50:38.574874 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.574833    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6qjl/must-gather-h477h"
Jul 28 16:50:38.574923 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.574890    2575 scope.go:117] "RemoveContainer" containerID="5b0db28ca6d021a431a408ae655348ce84fec8a509bfff2e005fe68aebc9937f"
Jul 28 16:50:38.583264 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.583243    2575 scope.go:117] "RemoveContainer" containerID="d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d"
Jul 28 16:50:38.596743 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.596719    2575 scope.go:117] "RemoveContainer" containerID="5b0db28ca6d021a431a408ae655348ce84fec8a509bfff2e005fe68aebc9937f"
Jul 28 16:50:38.597111 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:50:38.597064    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b0db28ca6d021a431a408ae655348ce84fec8a509bfff2e005fe68aebc9937f\": container with ID starting with 5b0db28ca6d021a431a408ae655348ce84fec8a509bfff2e005fe68aebc9937f not found: ID does not exist" containerID="5b0db28ca6d021a431a408ae655348ce84fec8a509bfff2e005fe68aebc9937f"
Jul 28 16:50:38.597234 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.597112    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b0db28ca6d021a431a408ae655348ce84fec8a509bfff2e005fe68aebc9937f"} err="failed to get container status \"5b0db28ca6d021a431a408ae655348ce84fec8a509bfff2e005fe68aebc9937f\": rpc error: code = NotFound desc = could not find container \"5b0db28ca6d021a431a408ae655348ce84fec8a509bfff2e005fe68aebc9937f\": container with ID starting with 5b0db28ca6d021a431a408ae655348ce84fec8a509bfff2e005fe68aebc9937f not found: ID does not exist"
Jul 28 16:50:38.597234 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.597133    2575 scope.go:117] "RemoveContainer" containerID="d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d"
Jul 28 16:50:38.597393 ip-10-0-140-224 kubenswrapper[2575]: E0728 16:50:38.597374    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d\": container with ID starting with d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d not found: ID does not exist" containerID="d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d"
Jul 28 16:50:38.597431 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.597400    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d"} err="failed to get container status \"d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d\": rpc error: code = NotFound desc = could not find container \"d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d\": container with ID starting with d9e7230a786852f7dd32f826b05b7c0145b0c8deb0daea8b77eb746f9489189d not found: ID does not exist"
Jul 28 16:50:38.647013 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.646975    2575 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff-must-gather-output\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:50:38.647013 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:38.647006    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vglvt\" (UniqueName: \"kubernetes.io/projected/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff-kube-api-access-vglvt\") on node \"ip-10-0-140-224.ec2.internal\" DevicePath \"\""
Jul 28 16:50:40.447037 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:40.447004    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" path="/var/lib/kubelet/pods/3366c9c7-d68b-4a25-b2d6-b12dea4f33ff/volumes"
Jul 28 16:50:40.902244 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:40.902213    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7c66fdcb8f-vk9c9_c11ce7a4-345c-4fe3-8fce-e240f1415048/metrics-server/0.log"
Jul 28 16:50:41.199287 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.199202    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wd94m_12b4dc4d-d95c-410b-9e4d-df1e16ecbda7/node-exporter/0.log"
Jul 28 16:50:41.259262 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.259237    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wd94m_12b4dc4d-d95c-410b-9e4d-df1e16ecbda7/kube-rbac-proxy/0.log"
Jul 28 16:50:41.300852 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.300824    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wd94m_12b4dc4d-d95c-410b-9e4d-df1e16ecbda7/init-textfile/0.log"
Jul 28 16:50:41.459973 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.459889    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6cae1800-bf89-4c00-a732-4a9a6b4de481/prometheus/0.log"
Jul 28 16:50:41.488291 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.488263    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6cae1800-bf89-4c00-a732-4a9a6b4de481/config-reloader/0.log"
Jul 28 16:50:41.518209 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.518178    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6cae1800-bf89-4c00-a732-4a9a6b4de481/thanos-sidecar/0.log"
Jul 28 16:50:41.547384 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.547361    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6cae1800-bf89-4c00-a732-4a9a6b4de481/kube-rbac-proxy-web/0.log"
Jul 28 16:50:41.575907 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.575879    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6cae1800-bf89-4c00-a732-4a9a6b4de481/kube-rbac-proxy/0.log"
Jul 28 16:50:41.607601 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.607569    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6cae1800-bf89-4c00-a732-4a9a6b4de481/kube-rbac-proxy-thanos/0.log"
Jul 28 16:50:41.636114 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.636070    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6cae1800-bf89-4c00-a732-4a9a6b4de481/init-config-reloader/0.log"
Jul 28 16:50:41.735843 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.735809    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-2gmnk_1936dcb1-1a31-4466-a3e7-c65e6dada324/prometheus-operator-admission-webhook/0.log"
Jul 28 16:50:41.772243 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.772214    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8574776b9-85vlk_81524ca0-40e5-4835-acd1-abe87263a284/telemeter-client/0.log"
Jul 28 16:50:41.797826 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.797795    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8574776b9-85vlk_81524ca0-40e5-4835-acd1-abe87263a284/reload/0.log"
Jul 28 16:50:41.827792 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:41.827762    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8574776b9-85vlk_81524ca0-40e5-4835-acd1-abe87263a284/kube-rbac-proxy/0.log"
Jul 28 16:50:44.578998 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.578964    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"]
Jul 28 16:50:44.579456 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.579239    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" containerName="gather"
Jul 28 16:50:44.579456 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.579250    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" containerName="gather"
Jul 28 16:50:44.579456 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.579263    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" containerName="copy"
Jul 28 16:50:44.579456 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.579269    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" containerName="copy"
Jul 28 16:50:44.579456 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.579277    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64a0de90-5b46-4266-a9db-34fe621656e9" containerName="switch-graph-d3a3d"
Jul 28 16:50:44.579456 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.579283    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="64a0de90-5b46-4266-a9db-34fe621656e9" containerName="switch-graph-d3a3d"
Jul 28 16:50:44.579456 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.579328    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="64a0de90-5b46-4266-a9db-34fe621656e9" containerName="switch-graph-d3a3d"
Jul 28 16:50:44.579456 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.579335    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" containerName="copy"
Jul 28 16:50:44.579456 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.579342    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3366c9c7-d68b-4a25-b2d6-b12dea4f33ff" containerName="gather"
Jul 28 16:50:44.583129 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.583083    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.585547 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.585493    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nzjbp\"/\"kube-root-ca.crt\""
Jul 28 16:50:44.585547 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.585533    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nzjbp\"/\"openshift-service-ca.crt\""
Jul 28 16:50:44.586053 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.586038    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nzjbp\"/\"default-dockercfg-llbc5\""
Jul 28 16:50:44.592007 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.591979    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"]
Jul 28 16:50:44.697779 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.697735    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/30ad3981-ffc6-4680-879b-7c6fcc475f67-proc\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.697779 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.697781    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30ad3981-ffc6-4680-879b-7c6fcc475f67-lib-modules\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.698008 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.697886    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/30ad3981-ffc6-4680-879b-7c6fcc475f67-podres\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.698008 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.697930    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30ad3981-ffc6-4680-879b-7c6fcc475f67-sys\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.698008 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.697958    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc46x\" (UniqueName: \"kubernetes.io/projected/30ad3981-ffc6-4680-879b-7c6fcc475f67-kube-api-access-vc46x\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.799265 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.799225    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/30ad3981-ffc6-4680-879b-7c6fcc475f67-podres\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.799385 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.799275    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30ad3981-ffc6-4680-879b-7c6fcc475f67-sys\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.799385 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.799312    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc46x\" (UniqueName: \"kubernetes.io/projected/30ad3981-ffc6-4680-879b-7c6fcc475f67-kube-api-access-vc46x\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.799385 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.799354    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/30ad3981-ffc6-4680-879b-7c6fcc475f67-proc\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.799385 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.799373    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30ad3981-ffc6-4680-879b-7c6fcc475f67-lib-modules\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.799564 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.799421    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/30ad3981-ffc6-4680-879b-7c6fcc475f67-podres\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.799564 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.799438    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30ad3981-ffc6-4680-879b-7c6fcc475f67-sys\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.799564 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.799474    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/30ad3981-ffc6-4680-879b-7c6fcc475f67-proc\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.799564 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.799516    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30ad3981-ffc6-4680-879b-7c6fcc475f67-lib-modules\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.808992 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.808963    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc46x\" (UniqueName: \"kubernetes.io/projected/30ad3981-ffc6-4680-879b-7c6fcc475f67-kube-api-access-vc46x\") pod \"perf-node-gather-daemonset-mgqt4\" (UID: \"30ad3981-ffc6-4680-879b-7c6fcc475f67\") " pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:44.893874 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:44.893781    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:45.028971 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:45.028933    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"]
Jul 28 16:50:45.032272 ip-10-0-140-224 kubenswrapper[2575]: W0728 16:50:45.032236    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod30ad3981_ffc6_4680_879b_7c6fcc475f67.slice/crio-128c4742f02bb55615c7158703257466d2a5f8b63c451d24fa0ecb8336897502 WatchSource:0}: Error finding container 128c4742f02bb55615c7158703257466d2a5f8b63c451d24fa0ecb8336897502: Status 404 returned error can't find the container with id 128c4742f02bb55615c7158703257466d2a5f8b63c451d24fa0ecb8336897502
Jul 28 16:50:45.603206 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:45.603165    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4" event={"ID":"30ad3981-ffc6-4680-879b-7c6fcc475f67","Type":"ContainerStarted","Data":"b447b0031b814b803c87c140884eed3cf2525801ae7c75c117d3fab6227a7df7"}
Jul 28 16:50:45.603678 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:45.603229    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:45.603678 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:45.603253    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4" event={"ID":"30ad3981-ffc6-4680-879b-7c6fcc475f67","Type":"ContainerStarted","Data":"128c4742f02bb55615c7158703257466d2a5f8b63c451d24fa0ecb8336897502"}
Jul 28 16:50:45.618380 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:45.618332    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4" podStartSLOduration=1.6183175109999999 podStartE2EDuration="1.618317511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:50:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:50:45.617279499 +0000 UTC m=+2971.761658081" watchObservedRunningTime="2026-07-28 16:50:45.618317511 +0000 UTC m=+2971.762696065"
Jul 28 16:50:46.170246 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:46.170216    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l8xg8_157a700c-ac88-481d-a050-7b2e3c0434e9/dns/0.log"
Jul 28 16:50:46.210143 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:46.210116    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l8xg8_157a700c-ac88-481d-a050-7b2e3c0434e9/kube-rbac-proxy/0.log"
Jul 28 16:50:46.297058 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:46.297027    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lbjpq_e3a33ca3-3c32-484a-88b7-8e8401f965fb/dns-node-resolver/0.log"
Jul 28 16:50:46.806913 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:46.806878    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-84b869bf79-7dm68_9cef1e5c-6531-4245-9341-c6c83e315dbe/registry/0.log"
Jul 28 16:50:46.958280 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:46.958250    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lkjfw_4d45a036-19ca-4d55-99d3-2264550e3891/node-ca/0.log"
Jul 28 16:50:48.321712 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:48.321681    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-scthp_60836054-f2ad-4604-a028-77b976a9cf5b/serve-healthcheck-canary/0.log"
Jul 28 16:50:48.844583 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:48.844552    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jdnzr_992fecb8-694b-4cf6-ae74-53488252c696/kube-rbac-proxy/0.log"
Jul 28 16:50:48.870478 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:48.870452    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jdnzr_992fecb8-694b-4cf6-ae74-53488252c696/exporter/0.log"
Jul 28 16:50:48.896487 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:48.896460    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jdnzr_992fecb8-694b-4cf6-ae74-53488252c696/extractor/0.log"
Jul 28 16:50:51.383070 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:51.383016    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-654c557856-8j76m_a2c555b7-83da-41ef-a8c7-358ac31a9dff/manager/0.log"
Jul 28 16:50:51.619142 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:51.619115    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nzjbp/perf-node-gather-daemonset-mgqt4"
Jul 28 16:50:51.749404 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:51.749369    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-kdj2d_717d4ef7-07af-496b-a727-dbdb172625de/s3-init/0.log"
Jul 28 16:50:51.770813 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:51.770785    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-kdj2d_717d4ef7-07af-496b-a727-dbdb172625de/download-hf-model/0.log"
Jul 28 16:50:57.930131 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:57.930083    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wnqjm_7a724adc-66b2-44b8-8284-216589688dd2/kube-multus-additional-cni-plugins/0.log"
Jul 28 16:50:57.960216 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:57.960183    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wnqjm_7a724adc-66b2-44b8-8284-216589688dd2/egress-router-binary-copy/0.log"
Jul 28 16:50:57.989961 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:57.989934    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wnqjm_7a724adc-66b2-44b8-8284-216589688dd2/cni-plugins/0.log"
Jul 28 16:50:58.015987 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:58.015950    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wnqjm_7a724adc-66b2-44b8-8284-216589688dd2/bond-cni-plugin/0.log"
Jul 28 16:50:58.043718 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:58.043689    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wnqjm_7a724adc-66b2-44b8-8284-216589688dd2/routeoverride-cni/0.log"
Jul 28 16:50:58.072447 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:58.072413    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wnqjm_7a724adc-66b2-44b8-8284-216589688dd2/whereabouts-cni-bincopy/0.log"
Jul 28 16:50:58.103877 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:58.103846    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wnqjm_7a724adc-66b2-44b8-8284-216589688dd2/whereabouts-cni/0.log"
Jul 28 16:50:58.217894 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:58.217860    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ll42f_f95682ff-9c53-4f22-a5ba-b2fa09f213f3/kube-multus/0.log"
Jul 28 16:50:58.305652 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:58.305622    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f2jjj_b549f8a9-6b3c-42bc-bbc8-11d57f9bab80/network-metrics-daemon/0.log"
Jul 28 16:50:58.332115 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:58.332067    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f2jjj_b549f8a9-6b3c-42bc-bbc8-11d57f9bab80/kube-rbac-proxy/0.log"
Jul 28 16:50:59.432219 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:59.432190    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hz6tj_76126427-9dc7-46c3-8120-b0704e288d20/ovn-controller/0.log"
Jul 28 16:50:59.456885 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:59.456856    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hz6tj_76126427-9dc7-46c3-8120-b0704e288d20/ovn-acl-logging/0.log"
Jul 28 16:50:59.482339 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:59.482306    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hz6tj_76126427-9dc7-46c3-8120-b0704e288d20/kube-rbac-proxy-node/0.log"
Jul 28 16:50:59.506205 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:59.506170    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hz6tj_76126427-9dc7-46c3-8120-b0704e288d20/kube-rbac-proxy-ovn-metrics/0.log"
Jul 28 16:50:59.531906 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:59.531881    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hz6tj_76126427-9dc7-46c3-8120-b0704e288d20/northd/0.log"
Jul 28 16:50:59.571219 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:59.571182    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hz6tj_76126427-9dc7-46c3-8120-b0704e288d20/nbdb/0.log"
Jul 28 16:50:59.635878 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:59.635850    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hz6tj_76126427-9dc7-46c3-8120-b0704e288d20/sbdb/0.log"
Jul 28 16:50:59.690853 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:50:59.690755    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hz6tj_76126427-9dc7-46c3-8120-b0704e288d20/ovnkube-controller/0.log"
Jul 28 16:51:01.092971 ip-10-0-140-224 kubenswrapper[2575]: I0728 16:51:01.092931    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-qknsz_bd6f7fa4-5752-4b05-a359-9277b293c7b6/network-check-target-container/0.log"