Jul 07 14:56:36.226410 ip-10-0-140-158 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.564291    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567608    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567619    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567622    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567625    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567628    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567632    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567635    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:36.610376 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567638    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567641    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567644    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567647    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567651    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567655    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567660    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567663    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567665    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567668    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567670    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567674    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567677    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567679    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567682    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567684    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567687    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567689    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567692    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:36.612606 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567695    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567697    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567700    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567702    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567707    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567710    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567713    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567716    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567718    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567721    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567724    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567726    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567729    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567732    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567734    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567737    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567740    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567744    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567746    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:36.615596 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567749    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567757    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567760    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567762    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567765    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567768    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567771    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567774    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567777    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567779    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567782    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567784    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567788    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567791    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567794    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567797    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567799    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567802    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567804    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567807    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:36.617469 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567810    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567813    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567815    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567818    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567820    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567823    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567825    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567828    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567830    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567833    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567836    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567839    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567842    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567844    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567847    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567850    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567852    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567855    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567857    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567860    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567863    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:36.618503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567866    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567868    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567871    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567877    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567879    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567882    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567884    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567887    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567889    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567892    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567894    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567897    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.567899    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568573    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568579    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568583    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568586    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568589    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568592    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:36.625792 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568594    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568597    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568599    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568603    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568605    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568608    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568611    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568613    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568616    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568619    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568621    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568624    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568627    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568629    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568631    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568634    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568637    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568640    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568642    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568645    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:36.628503 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568647    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568650    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568653    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568655    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568660    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568664    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568667    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568670    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568673    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568676    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568678    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568681    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568683    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568686    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568689    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568692    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568695    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568698    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568701    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:36.632689 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568703    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:36.631421 ip-10-0-140-158 systemd[1]: Started Kubernetes Kubelet.
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568706    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568708    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568711    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568713    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568716    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568719    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568721    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568726    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568729    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568732    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568734    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568737    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568739    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568742    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568744    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568747    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568749    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568751    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568754    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:36.633996 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568756    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568759    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568761    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568763    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568766    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568768    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568770    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568773    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568776    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568779    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568782    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568785    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568787    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568790    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568793    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568795    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568798    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568801    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568803    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568806    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:36.636874 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568809    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568811    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568813    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568816    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568818    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568821    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568823    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568825    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568828    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568833    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568836    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568839    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568841    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.568844    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569195    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569200    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569204    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569206    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569209    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569211    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:36.637684 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569214    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569218    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569222    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569225    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569228    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569231    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569233    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569237    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569239    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569242    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569244    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569247    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569250    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569252    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569255    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569258    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569261    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569264    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569266    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:36.638288 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569269    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569271    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569277    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569279    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569282    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569284    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569287    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569290    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569293    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569295    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569298    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569300    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569303    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569305    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569308    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569310    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569313    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569316    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569318    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569321    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:36.639093 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569324    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569327    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569329    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569332    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569335    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569339    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569341    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569344    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569347    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569349    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569352    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569355    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569357    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569359    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569364    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569366    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569368    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569371    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569373    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:36.639654 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569376    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569378    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569381    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569384    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569387    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569389    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569392    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569394    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569397    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569399    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569402    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569405    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569407    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569410    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569412    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569415    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569417    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569420    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569422    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569425    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:36.640217 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569429    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569431    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569435    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569439    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569442    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569445    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569448    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569452    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569455    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569457    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569460    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569463    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569465    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569468    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.569470    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569530    2582 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569537    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569543    2582 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569547    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569551    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 14:56:36.640744 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569554    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569558    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569562    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569566    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569569    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569572    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569575    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569578    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569581    2582 flags.go:64] FLAG: --cgroup-root=""
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569584    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569587    2582 flags.go:64] FLAG: --client-ca-file=""
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569589    2582 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569594    2582 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569598    2582 flags.go:64] FLAG: --cluster-domain=""
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569600    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569603    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569607    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569610    2582 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569613    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569618    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569621    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569624    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569627    2582 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569629    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 14:56:36.641460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569632    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569635    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569638    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569642    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569645    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569647    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569650    2582 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569653    2582 flags.go:64] FLAG: --enable-server="true"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569656    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569661    2582 flags.go:64] FLAG: --event-burst="100"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569679    2582 flags.go:64] FLAG: --event-qps="50"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569683    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569686    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569689    2582 flags.go:64] FLAG: --eviction-hard=""
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569693    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569696    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569699    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569702    2582 flags.go:64] FLAG: --eviction-soft=""
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569705    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569708    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569712    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569715    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569718    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569721    2582 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569724    2582 flags.go:64] FLAG: --feature-gates=""
Jul 07 14:56:36.642234 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569727    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569730    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569734    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569737    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569740    2582 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569744    2582 flags.go:64] FLAG: --help="false"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569746    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-140-158.ec2.internal"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569750    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569753    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569755    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569759    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569762    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569765    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569767    2582 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569770    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569773    2582 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569776    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569779    2582 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569782    2582 flags.go:64] FLAG: --kube-reserved=""
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569785    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569788    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569791    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569793    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569796    2582 flags.go:64] FLAG: --lock-file=""
Jul 07 14:56:36.643339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569799    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569801    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569804    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569811    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569814    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569817    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569819    2582 flags.go:64] FLAG: --logging-format="text"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569822    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569825    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569828    2582 flags.go:64] FLAG: --manifest-url=""
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569832    2582 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569839    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569842    2582 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569846    2582 flags.go:64] FLAG: --max-pods="110"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569849    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569852    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569855    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569858    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569861    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569863    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569866    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569872    2582 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569875    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569878    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 14:56:36.644428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569881    2582 flags.go:64] FLAG: --pod-cidr=""
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569884    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569889    2582 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569892    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569895    2582 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569899    2582 flags.go:64] FLAG: --port="10250"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569902    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569904    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-096371a855c5f5740"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569908    2582 flags.go:64] FLAG: --qos-reserved=""
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569910    2582 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569913    2582 flags.go:64] FLAG: --register-node="true"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569918    2582 flags.go:64] FLAG: --register-with-taints=""
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569921    2582 flags.go:64] FLAG: --registry-burst="10"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569924    2582 flags.go:64] FLAG: --registry-qps="5"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569927    2582 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569929    2582 flags.go:64] FLAG: --reserved-memory=""
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569933    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569936    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569938    2582 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569943    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569945    2582 flags.go:64] FLAG: --runonce="false"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569948    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569951    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569954    2582 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569957    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 14:56:36.645321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569960    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569963    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569966    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569969    2582 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569972    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569975    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569978    2582 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569980    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569983    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569986    2582 flags.go:64] FLAG: --system-cgroups=""
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569989    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569994    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.569997    2582 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.570000    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.570004    2582 flags.go:64] FLAG: --tls-min-version=""
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.570006    2582 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.570009    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.570012    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.570016    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.570018    2582 flags.go:64] FLAG: --v="2"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.570022    2582 flags.go:64] FLAG: --version="false"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.570026    2582 flags.go:64] FLAG: --vmodule=""
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.570029    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.570033    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570154    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:36.646434 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570158    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570164    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570168    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570171    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570174    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570177    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570180    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570182    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570184    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570187    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570190    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570193    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570195    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570197    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570200    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570202    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570205    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570208    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570210    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570213    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:36.647370 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570216    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570219    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570221    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570224    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570226    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570230    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570232    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570235    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570238    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570240    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570243    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570245    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570248    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570251    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570254    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570256    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570259    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570261    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570264    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:36.648403 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570267    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570270    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570274    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570277    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570279    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570282    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570284    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570287    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570289    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570292    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570295    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570297    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570300    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570302    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570305    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570308    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570311    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570313    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570317    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570320    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:36.649527 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570322    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570324    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570327    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570330    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570333    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570335    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570339    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570342    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570344    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570347    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570349    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570353    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570357    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570360    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570362    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570365    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570368    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570371    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570373    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:36.650902 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570375    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570378    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570381    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570383    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570386    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570388    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570391    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570393    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570396    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570403    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570405    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570409    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570412    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570415    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570417    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570420    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570423    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570425    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570428    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:36.651966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.570432    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.571092    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.578960    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.578975    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.578999    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.579006    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579070    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579078    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579082    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579085    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579088    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579091    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579093    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579097    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:36.653043 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579099    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579103    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579106    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579108    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579111    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579114    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579116    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579119    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579121    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579135    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579139    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579144    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579148    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579152    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579159    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579164    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579167    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579170    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579174    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:36.653942 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579177    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579180    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579183    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579185    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579188    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579191    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579193    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579196    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579199    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579201    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579204    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579207    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579209    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579211    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579214    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579217    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579220    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579225    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579229    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:36.654979 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579234    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579239    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579241    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579244    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579247    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579249    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579252    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579254    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579260    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579263    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579265    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579268    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579271    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579275    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579277    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579280    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579282    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579287    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579290    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:36.842856 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579294    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579298    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579302    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579307    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579311    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579316    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579320    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579323    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579326    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579328    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579331    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579333    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579336    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579338    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579342    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579344    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579347    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579349    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579352    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579354    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:36.864691 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579357    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579361    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579363    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579366    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579369    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579371    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579375    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579377    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579380    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579383    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579387    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579392    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579396    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579400    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.579405    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:56:36.865360 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579539    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579546    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579551    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579555    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579560    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579566    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579570    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579573    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579577    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579579    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579582    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579585    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579590    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579592    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579595    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579598    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579600    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579603    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579606    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:36.865907 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579610    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579613    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579615    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579619    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579624    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579629    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579634    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579639    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579644    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579648    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579651    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579653    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579656    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579659    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579661    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579664    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579666    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579669    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579672    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:36.866557 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579674    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579677    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579681    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579684    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579686    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579689    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579691    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579694    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579697    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579699    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579702    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579704    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579707    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579713    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579717    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579721    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579725    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579729    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579732    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579734    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:36.867179 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579737    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579739    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579742    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579745    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579747    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579750    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579752    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579755    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579757    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579760    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579762    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579765    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579767    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579770    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579772    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579775    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579777    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579780    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579782    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579785    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:36.867891 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579789    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579793    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579797    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579801    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579805    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579810    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579813    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579816    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579818    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579822    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579825    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579827    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579830    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579832    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579835    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579837    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579840    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579842    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579845    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579847    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:36.868491 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:36.579850    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:36.869081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.579855    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:56:36.869081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.579998    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 14:56:36.869081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.582115    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 14:56:36.869081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.583352    2582 server.go:1019] "Starting client certificate rotation"
Jul 07 14:56:36.869081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.583457    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 14:56:36.869081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.583491    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 14:56:36.869081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.585507    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 14:56:36.869081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.587632    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 14:56:36.869081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.598741    2582 log.go:25] "Validated CRI v1 runtime API"
Jul 07 14:56:36.869081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.601334    2582 log.go:25] "Validated CRI v1 image API"
Jul 07 14:56:36.869081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.602185    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 14:56:36.869081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.604467    2582 fs.go:135] Filesystem UUIDs: map[08d30656-16a9-4eaa-9836-d6001a9f0aaf:/dev/nvme0n1p3 5b6854f0-297c-4329-8e90-8b5c5d9beb74:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 07 14:56:36.869501 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.604483    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 14:56:36.869501 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.610925    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.611312    2582 manager.go:216] Machine: {Timestamp:2026-07-07 14:56:36.610119139 +0000 UTC m=+0.283725185 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100167 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b08928c24433842e67dbcc37c8929 SystemUUID:ec2b0892-8c24-4338-42e6-7dbcc37c8929 BootID:9250e28b-b12b-46b2-a6e7-67ae99c6908d Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:38:78:c3:c3:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:38:78:c3:c3:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:9e:c7:53:b1:5c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.611419    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.611495    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.613934    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.613959    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-158.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.615067    2582 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.615079    2582 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.615089    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.615713    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.617012    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.620711    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.620844    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.620878    2582 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.620893    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.620910    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.620925    2582 kubelet.go:398] "Adding apiserver pod source"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.620936    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 14:56:36.869737 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.621366    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.621412    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.623308    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.625304    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.625333    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.626445    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.626463    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.626471    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.626484    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.626495    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.626518    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.626531    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.626538    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.626549    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.626556    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.626569    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.626581    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.627192    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.627200    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.630710    2582 server.go:1268] "Started kubelet"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.631115    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.632170    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.632224    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.633029    2582 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.633429    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.633833    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-158.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.634486    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-158.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.634876    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.637545    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.637273    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-158.ec2.internal.18c009c593be340d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-158.ec2.internal,UID:ip-10-0-140-158.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-158.ec2.internal,},FirstTimestamp:2026-07-07 14:56:36.630639629 +0000 UTC m=+0.304245674,LastTimestamp:2026-07-07 14:56:36.630639629 +0000 UTC m=+0.304245674,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-158.ec2.internal,}"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.641319    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.641387    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-158.ec2.internal\" not found"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.641956    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.642108    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.642195    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.642310    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.642464    2582 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.642475    2582 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.643886    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.643901    2582 factory.go:55] Registering systemd factory
Jul 07 14:56:36.870357 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.643909    2582 factory.go:223] Registration of the systemd container factory successfully
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.644191    2582 factory.go:153] Registering CRI-O factory
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.644207    2582 factory.go:223] Registration of the crio container factory successfully
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.644229    2582 factory.go:103] Registering Raw factory
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.644245    2582 manager.go:1196] Started watching for new ooms in manager
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.644637    2582 manager.go:349] Starting recovery of all containers
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.650488    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-szrcq"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.655782    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.655954    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-158.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.656046    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.656556    2582 manager.go:354] Recovery completed
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.658720    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-szrcq"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.661460    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.663400    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.663435    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.663445    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.663883    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.663891    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.663904    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.663945    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.664578    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.664598    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.664607    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.665422    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-158.ec2.internal.18c009c595b26c52  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-158.ec2.internal,UID:ip-10-0-140-158.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-158.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-158.ec2.internal,},FirstTimestamp:2026-07-07 14:56:36.663422034 +0000 UTC m=+0.337028080,LastTimestamp:2026-07-07 14:56:36.663422034 +0000 UTC m=+0.337028080,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-158.ec2.internal,}"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.665993    2582 policy_none.go:49] "None policy: Start"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.666006    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.666016    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.670734    2582 policy_none.go:47] "Start"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.710705    2582 manager.go:337] "Starting Device Plugin manager"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.710861    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.710876    2582 server.go:87] "Starting device plugin registration server"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.711148    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.711159    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.711244    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.711330    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.711339    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.711854    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.711920    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-158.ec2.internal\" not found"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.786355    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.786398    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.786456    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 14:56:36.871681 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.786498    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 14:56:36.873895 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.789160    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:36.873895 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.812458    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:36.873895 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.813297    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:36.873895 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.813323    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:36.873895 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.813333    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:36.873895 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.813368    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-158.ec2.internal"
Jul 07 14:56:36.873895 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.824801    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-158.ec2.internal"
Jul 07 14:56:36.873895 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.824825    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-158.ec2.internal\": node \"ip-10-0-140-158.ec2.internal\" not found"
Jul 07 14:56:36.873895 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.845773    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-158.ec2.internal\" not found"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.887289    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-158.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal"]
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.887357    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.888385    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.888414    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.888428    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.890901    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.890986    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.891020    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.891633    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.891657    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.891667    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.891708    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.891728    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.891737    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.893930    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.893957    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.894572    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.894594    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.894607    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.921688    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-158.ec2.internal\" not found" node="ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.926043    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-158.ec2.internal\" not found" node="ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.944924    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7649a49abd2efd166fe10d91dbf1337c-config\") pod \"kube-apiserver-proxy-ip-10-0-140-158.ec2.internal\" (UID: \"7649a49abd2efd166fe10d91dbf1337c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.944944    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2933e68aa89922301a1452efadf03267-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal\" (UID: \"2933e68aa89922301a1452efadf03267\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:36.944964    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2933e68aa89922301a1452efadf03267-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal\" (UID: \"2933e68aa89922301a1452efadf03267\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.012981 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:36.945842    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-158.ec2.internal\" not found"
Jul 07 14:56:37.075688 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.045338    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7649a49abd2efd166fe10d91dbf1337c-config\") pod \"kube-apiserver-proxy-ip-10-0-140-158.ec2.internal\" (UID: \"7649a49abd2efd166fe10d91dbf1337c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.075688 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.045373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2933e68aa89922301a1452efadf03267-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal\" (UID: \"2933e68aa89922301a1452efadf03267\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.075688 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.045399    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2933e68aa89922301a1452efadf03267-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal\" (UID: \"2933e68aa89922301a1452efadf03267\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.075688 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.045438    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7649a49abd2efd166fe10d91dbf1337c-config\") pod \"kube-apiserver-proxy-ip-10-0-140-158.ec2.internal\" (UID: \"7649a49abd2efd166fe10d91dbf1337c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.075688 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.045453    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2933e68aa89922301a1452efadf03267-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal\" (UID: \"2933e68aa89922301a1452efadf03267\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.075688 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.045460    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2933e68aa89922301a1452efadf03267-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal\" (UID: \"2933e68aa89922301a1452efadf03267\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.075688 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:37.046138    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-158.ec2.internal\" not found"
Jul 07 14:56:37.272740 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:37.146933    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-158.ec2.internal\" not found"
Jul 07 14:56:37.272740 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.228486    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.272740 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.229628    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.272740 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:37.248101    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-158.ec2.internal\" not found"
Jul 07 14:56:37.461856 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:37.348887    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-158.ec2.internal\" not found"
Jul 07 14:56:37.461856 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:37.449606    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-158.ec2.internal\" not found"
Jul 07 14:56:37.653348 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:37.550289    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-158.ec2.internal\" not found"
Jul 07 14:56:37.653348 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.582790    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 14:56:37.653348 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.582959    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 14:56:37.653348 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:37.650749    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-158.ec2.internal\" not found"
Jul 07 14:56:37.810971 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.656132    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:37.810971 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.663710    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 14:51:37 +0000 UTC" deadline="2028-02-13 13:54:30.393026009 +0000 UTC"
Jul 07 14:56:37.810971 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.663734    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14062h57m52.729294849s"
Jul 07 14:56:37.810971 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.742877    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.810971 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.754749    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 14:56:37.810971 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.756267    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal"
Jul 07 14:56:37.810971 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.773091    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 14:56:37.967229 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:37.951500    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7649a49abd2efd166fe10d91dbf1337c.slice/crio-9e29e5f35a5fcec68471a6b76b86646aa73b44bba53be9a76e9178bb07263a2b WatchSource:0}: Error finding container 9e29e5f35a5fcec68471a6b76b86646aa73b44bba53be9a76e9178bb07263a2b: Status 404 returned error can't find the container with id 9e29e5f35a5fcec68471a6b76b86646aa73b44bba53be9a76e9178bb07263a2b
Jul 07 14:56:37.967229 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:37.951822    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2933e68aa89922301a1452efadf03267.slice/crio-7ad65341b30adbdcf5706dcaf19fbfca11fd1b203d8f4a72240b9b31c01a0000 WatchSource:0}: Error finding container 7ad65341b30adbdcf5706dcaf19fbfca11fd1b203d8f4a72240b9b31c01a0000: Status 404 returned error can't find the container with id 7ad65341b30adbdcf5706dcaf19fbfca11fd1b203d8f4a72240b9b31c01a0000
Jul 07 14:56:37.967229 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:37.956303    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 14:56:38.008951 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.008922    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:38.199730 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.199697    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:38.622945 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.622913    2582 apiserver.go:52] "Watching apiserver"
Jul 07 14:56:38.630710 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.630682    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 14:56:38.631995 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.631964    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal","openshift-network-diagnostics/network-check-target-7dqwt","openshift-network-operator/iptables-alerter-mbjwr","openshift-ovn-kubernetes/ovnkube-node-x9bhb","kube-system/global-pull-secret-syncer-266nr","kube-system/kube-apiserver-proxy-ip-10-0-140-158.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr","openshift-dns/node-resolver-x9g98","openshift-multus/multus-9k67d","openshift-multus/multus-additional-cni-plugins-xdxzl","openshift-multus/network-metrics-daemon-dzg7v","kube-system/konnectivity-agent-cfssq","openshift-cluster-node-tuning-operator/tuned-7s7fg","openshift-image-registry/node-ca-gnn6l"]
Jul 07 14:56:38.635003 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.634979    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.637751 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.637725    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.638754 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.638732    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 14:56:38.639751 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.639123    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sp2xw\""
Jul 07 14:56:38.639751 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.639351    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 14:56:38.639751 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.639611    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 14:56:38.640417 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.640286    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mbjwr"
Jul 07 14:56:38.643229 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.642327    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jgx76\""
Jul 07 14:56:38.643229 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.643021    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:56:38.651130 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.648872    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 14:56:38.651130 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.649604    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.651130 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.650481    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 14:56:38.652066 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.651715    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:56:38.652066 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.651831    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 14:56:38.652066 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.651727    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 14:56:38.654432 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.654409    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:38.655015 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:38.654499    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:56:38.655083 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.655040    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-w748q\""
Jul 07 14:56:38.655332 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.655310    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.655630 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.655605    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-registration-dir\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.655712 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.655654    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-sys\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.655712 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.655692    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-var-lib-kubelet\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.655823 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.655726    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg94f\" (UniqueName: \"kubernetes.io/projected/75ba721c-5d55-4692-ae73-5cd48df6d08c-kube-api-access-gg94f\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.655823 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.655762    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27rk2\" (UniqueName: \"kubernetes.io/projected/98536497-3355-4ae8-84c4-cbf6a100056b-kube-api-access-27rk2\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.655823 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.655797    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-lib-modules\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.655973 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.655823    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-host\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.655973 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.655855    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b33fcf01-5654-4cff-a8df-b7294a3f712f-iptables-alerter-script\") pod \"iptables-alerter-mbjwr\" (UID: \"b33fcf01-5654-4cff-a8df-b7294a3f712f\") " pod="openshift-network-operator/iptables-alerter-mbjwr"
Jul 07 14:56:38.656076 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.656006    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 14:56:38.656129 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.656085    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 14:56:38.656716 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.655889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-var-lib-openvswitch\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.656800 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.656780    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-run-openvswitch\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.656861 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.656817    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7mkvp\""
Jul 07 14:56:38.656861 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.656819    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-node-log\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.656955 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.656863    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-tuned\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.656955 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.656886    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75ba721c-5d55-4692-ae73-5cd48df6d08c-tmp\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.656955 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.656908    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b33fcf01-5654-4cff-a8df-b7294a3f712f-host-slash\") pod \"iptables-alerter-mbjwr\" (UID: \"b33fcf01-5654-4cff-a8df-b7294a3f712f\") " pod="openshift-network-operator/iptables-alerter-mbjwr"
Jul 07 14:56:38.656955 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.656936    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657145 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.656964    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/54a7480b-096b-4bdf-8352-632d0216e013-ovnkube-config\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657145 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657010    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/54a7480b-096b-4bdf-8352-632d0216e013-env-overrides\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657145 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657037    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-sysctl-d\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.657145 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657071    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-systemd\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.657145 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657097    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47k6s\" (UniqueName: \"kubernetes.io/projected/b33fcf01-5654-4cff-a8df-b7294a3f712f-kube-api-access-47k6s\") pod \"iptables-alerter-mbjwr\" (UID: \"b33fcf01-5654-4cff-a8df-b7294a3f712f\") " pod="openshift-network-operator/iptables-alerter-mbjwr"
Jul 07 14:56:38.657145 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657121    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-systemd-units\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657448 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657150    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-slash\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657448 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657197    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-run-systemd\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657448 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657222    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-etc-openvswitch\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657448 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657246    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/54a7480b-096b-4bdf-8352-632d0216e013-ovnkube-script-lib\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657448 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657271    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-run\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.657448 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh467\" (UniqueName: \"kubernetes.io/projected/54a7480b-096b-4bdf-8352-632d0216e013-kube-api-access-dh467\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657448 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657346    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-sysconfig\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.657448 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657370    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-kubernetes\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.657448 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657378    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 14:56:38.657448 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657397    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-sysctl-conf\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.657448 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657423    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-run-ovn-kubernetes\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657464    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-cni-bin\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657506    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-etc-selinux\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657560    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-sys-fs\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657565    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657595    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-modprobe-d\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657621    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-log-socket\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657666    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-socket-dir\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657699    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-device-dir\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657736    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-kubelet\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657762    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-run-netns\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657788    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-run-ovn\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657822    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gnn6l"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657836    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-cni-netd\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.657960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.657860    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/54a7480b-096b-4bdf-8352-632d0216e013-ovn-node-metrics-cert\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.658550 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.658291    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 14:56:38.658550 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.658393    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 14:56:38.660297 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.660274    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zctwf\""
Jul 07 14:56:38.660646 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.660628    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.662728 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.662708    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 14:56:38.662887 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.662719    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 14:56:38.662990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.662867    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x9g98"
Jul 07 14:56:38.663759 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.663657    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 14:56:38.664417 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.664391    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:38.664497 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.664480    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 14:56:38.664573 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.664500    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 14:56:38.664760 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.664741    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 14:56:38.665069 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.665051    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 14:56:38.665236 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.665159    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mlrxl\""
Jul 07 14:56:38.665479 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.665460    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:38.665585 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:38.665558    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:56:38.666231 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.666212    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 14:56:38.666431 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.666414    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 14:56:38.666506 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.666486    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9jnxw\""
Jul 07 14:56:38.668137 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.668115    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.670885 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.670863    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:38.670990 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:38.670960    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:56:38.672190 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.671452    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 14:56:38.672190 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.671542    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zkj7z\""
Jul 07 14:56:38.675839 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.672400    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 14:56:38.675839 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.672820    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 14:56:38.675839 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.673464    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cfssq"
Jul 07 14:56:38.679029 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.678040    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 14:56:38.679029 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.678301    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 14:56:38.679029 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.678626    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xxpw4\""
Jul 07 14:56:38.696569 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.696546    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cv4jk"
Jul 07 14:56:38.703457 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.703434    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cv4jk"
Jul 07 14:56:38.746566 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.745471    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 14:56:38.758971 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.758937    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-systemd\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.759100 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.758979    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47k6s\" (UniqueName: \"kubernetes.io/projected/b33fcf01-5654-4cff-a8df-b7294a3f712f-kube-api-access-47k6s\") pod \"iptables-alerter-mbjwr\" (UID: \"b33fcf01-5654-4cff-a8df-b7294a3f712f\") " pod="openshift-network-operator/iptables-alerter-mbjwr"
Jul 07 14:56:38.759100 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759007    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-slash\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.759100 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759033    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-run-systemd\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.759100 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759058    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-etc-openvswitch\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.759100 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759083    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh467\" (UniqueName: \"kubernetes.io/projected/54a7480b-096b-4bdf-8352-632d0216e013-kube-api-access-dh467\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.759100 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759082    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-systemd\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.759100 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759082    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-slash\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759112    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-var-lib-kubelet\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759126    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-run-systemd\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759167    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-etc-openvswitch\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759203    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-multus-daemon-config\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759226    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-cni-binary-copy\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759253    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-cnibin\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759276    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-var-lib-cni-multus\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759301    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/caee8dda-918b-405f-af0c-f7bb812ab885-hosts-file\") pod \"node-resolver-x9g98\" (UID: \"caee8dda-918b-405f-af0c-f7bb812ab885\") " pod="openshift-dns/node-resolver-x9g98"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759332    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-sysconfig\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759372    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-kubernetes\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759389    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-sysconfig\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759429    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-sysctl-conf\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.759495 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759448    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-kubernetes\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759529    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-run-ovn-kubernetes\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759597    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-sysctl-conf\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759583    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-run-ovn-kubernetes\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759620    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c-agent-certs\") pod \"konnectivity-agent-cfssq\" (UID: \"3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c\") " pod="kube-system/konnectivity-agent-cfssq"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759648    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caee8dda-918b-405f-af0c-f7bb812ab885-tmp-dir\") pod \"node-resolver-x9g98\" (UID: \"caee8dda-918b-405f-af0c-f7bb812ab885\") " pod="openshift-dns/node-resolver-x9g98"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759665    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfbv5\" (UniqueName: \"kubernetes.io/projected/caee8dda-918b-405f-af0c-f7bb812ab885-kube-api-access-cfbv5\") pod \"node-resolver-x9g98\" (UID: \"caee8dda-918b-405f-af0c-f7bb812ab885\") " pod="openshift-dns/node-resolver-x9g98"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759691    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-sys-fs\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759731    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759785    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-sys-fs\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759797    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759838    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c-konnectivity-ca\") pod \"konnectivity-agent-cfssq\" (UID: \"3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c\") " pod="kube-system/konnectivity-agent-cfssq"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759872    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-run-k8s-cni-cncf-io\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759906    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-socket-dir\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759931    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-device-dir\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-kubelet\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.759979    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-run-ovn\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.760258 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760007    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/54a7480b-096b-4bdf-8352-632d0216e013-ovn-node-metrics-cert\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760031    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-cnibin\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760049    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-multus-conf-dir\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760065    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760054    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-socket-dir\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-registration-dir\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760102    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-sys\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760118    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-var-lib-kubelet\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760135    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg94f\" (UniqueName: \"kubernetes.io/projected/75ba721c-5d55-4692-ae73-5cd48df6d08c-kube-api-access-gg94f\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760151    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760187    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bde44302-557d-46b5-9d01-c43bcc78d2ec-kubelet-config\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760204    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bde44302-557d-46b5-9d01-c43bcc78d2ec-dbus\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760222    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27rk2\" (UniqueName: \"kubernetes.io/projected/98536497-3355-4ae8-84c4-cbf6a100056b-kube-api-access-27rk2\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760239    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-lib-modules\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760254    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-var-lib-openvswitch\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-node-log\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.761159 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760284    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-os-release\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760307    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ljlz\" (UniqueName: \"kubernetes.io/projected/c132d28e-79ff-4bef-8d61-b5a24f10a279-kube-api-access-4ljlz\") pod \"node-ca-gnn6l\" (UID: \"c132d28e-79ff-4bef-8d61-b5a24f10a279\") " pod="openshift-image-registry/node-ca-gnn6l"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760329    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-multus-socket-dir-parent\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760345    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-tuned\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760365    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75ba721c-5d55-4692-ae73-5cd48df6d08c-tmp\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760389    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b33fcf01-5654-4cff-a8df-b7294a3f712f-host-slash\") pod \"iptables-alerter-mbjwr\" (UID: \"b33fcf01-5654-4cff-a8df-b7294a3f712f\") " pod="openshift-network-operator/iptables-alerter-mbjwr"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/54a7480b-096b-4bdf-8352-632d0216e013-ovnkube-config\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c132d28e-79ff-4bef-8d61-b5a24f10a279-host\") pod \"node-ca-gnn6l\" (UID: \"c132d28e-79ff-4bef-8d61-b5a24f10a279\") " pod="openshift-image-registry/node-ca-gnn6l"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760435    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760518    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760562    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-registration-dir\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760600    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-sys\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760661    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-var-lib-kubelet\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760775    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-kubelet\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760781    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-var-lib-openvswitch\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760901    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-node-log\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760929    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-lib-modules\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760445    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-os-release\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.761990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.760988    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-device-dir\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761022    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvdm4\" (UniqueName: \"kubernetes.io/projected/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-kube-api-access-jvdm4\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761031    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b33fcf01-5654-4cff-a8df-b7294a3f712f-host-slash\") pod \"iptables-alerter-mbjwr\" (UID: \"b33fcf01-5654-4cff-a8df-b7294a3f712f\") " pod="openshift-network-operator/iptables-alerter-mbjwr"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761063    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-sysctl-d\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761070    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-run-ovn\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761102    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-systemd-units\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761147    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/54a7480b-096b-4bdf-8352-632d0216e013-ovnkube-script-lib\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-etc-kubernetes\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761226    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-sysctl-d\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-run\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761416    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-hostroot\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761446    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-run-multus-certs\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-cni-bin\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761495    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-system-cni-dir\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761517    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-multus-cni-dir\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761536    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-var-lib-cni-bin\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761560    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-etc-selinux\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.762939 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761584    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-modprobe-d\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761609    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-log-socket\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761632    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wl8q\" (UniqueName: \"kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q\") pod \"network-check-target-7dqwt\" (UID: \"b8bd0203-d82d-4727-b612-ef4669b4eab3\") " pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761653    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtlkd\" (UniqueName: \"kubernetes.io/projected/b0469d85-dc53-4bb1-891e-99ed1d55fee8-kube-api-access-jtlkd\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761675    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-run-netns\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761678    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-systemd-units\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761696    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-run-netns\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-cni-netd\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761739    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c132d28e-79ff-4bef-8d61-b5a24f10a279-serviceca\") pod \"node-ca-gnn6l\" (UID: \"c132d28e-79ff-4bef-8d61-b5a24f10a279\") " pod="openshift-image-registry/node-ca-gnn6l"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761781    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-run\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761792    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-system-cni-dir\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761819    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd8nw\" (UniqueName: \"kubernetes.io/projected/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-kube-api-access-bd8nw\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761842    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-cni-bin\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761845    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-cni-binary-copy\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761885    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-host\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761913    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b33fcf01-5654-4cff-a8df-b7294a3f712f-iptables-alerter-script\") pod \"iptables-alerter-mbjwr\" (UID: \"b33fcf01-5654-4cff-a8df-b7294a3f712f\") " pod="openshift-network-operator/iptables-alerter-mbjwr"
Jul 07 14:56:38.763817 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761946    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/54a7480b-096b-4bdf-8352-632d0216e013-ovnkube-script-lib\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-run-openvswitch\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761966    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98536497-3355-4ae8-84c4-cbf6a100056b-etc-selinux\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761983    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.762028    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.762047    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-modprobe-d\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.762069    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-run-openvswitch\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.762091    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-log-socket\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.762113    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75ba721c-5d55-4692-ae73-5cd48df6d08c-host\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.761630    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/54a7480b-096b-4bdf-8352-632d0216e013-ovnkube-config\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.762169    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-cni-netd\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.762199    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54a7480b-096b-4bdf-8352-632d0216e013-host-run-netns\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.762262    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/54a7480b-096b-4bdf-8352-632d0216e013-env-overrides\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.762290    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.762505    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b33fcf01-5654-4cff-a8df-b7294a3f712f-iptables-alerter-script\") pod \"iptables-alerter-mbjwr\" (UID: \"b33fcf01-5654-4cff-a8df-b7294a3f712f\") " pod="openshift-network-operator/iptables-alerter-mbjwr"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.762722    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/54a7480b-096b-4bdf-8352-632d0216e013-env-overrides\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.764402    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75ba721c-5d55-4692-ae73-5cd48df6d08c-etc-tuned\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.764631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.764495    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75ba721c-5d55-4692-ae73-5cd48df6d08c-tmp\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.765353 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.764878    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/54a7480b-096b-4bdf-8352-632d0216e013-ovn-node-metrics-cert\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.774471 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.774445    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh467\" (UniqueName: \"kubernetes.io/projected/54a7480b-096b-4bdf-8352-632d0216e013-kube-api-access-dh467\") pod \"ovnkube-node-x9bhb\" (UID: \"54a7480b-096b-4bdf-8352-632d0216e013\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:38.774947 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.774924    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27rk2\" (UniqueName: \"kubernetes.io/projected/98536497-3355-4ae8-84c4-cbf6a100056b-kube-api-access-27rk2\") pod \"aws-ebs-csi-driver-node-9ckqr\" (UID: \"98536497-3355-4ae8-84c4-cbf6a100056b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.775417 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.775393    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47k6s\" (UniqueName: \"kubernetes.io/projected/b33fcf01-5654-4cff-a8df-b7294a3f712f-kube-api-access-47k6s\") pod \"iptables-alerter-mbjwr\" (UID: \"b33fcf01-5654-4cff-a8df-b7294a3f712f\") " pod="openshift-network-operator/iptables-alerter-mbjwr"
Jul 07 14:56:38.776113 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.776094    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg94f\" (UniqueName: \"kubernetes.io/projected/75ba721c-5d55-4692-ae73-5cd48df6d08c-kube-api-access-gg94f\") pod \"tuned-7s7fg\" (UID: \"75ba721c-5d55-4692-ae73-5cd48df6d08c\") " pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.791846 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.791796    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal" event={"ID":"2933e68aa89922301a1452efadf03267","Type":"ContainerStarted","Data":"7ad65341b30adbdcf5706dcaf19fbfca11fd1b203d8f4a72240b9b31c01a0000"}
Jul 07 14:56:38.793510 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.793485    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-158.ec2.internal" event={"ID":"7649a49abd2efd166fe10d91dbf1337c","Type":"ContainerStarted","Data":"9e29e5f35a5fcec68471a6b76b86646aa73b44bba53be9a76e9178bb07263a2b"}
Jul 07 14:56:38.863141 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863088    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:38.863304 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863135    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c132d28e-79ff-4bef-8d61-b5a24f10a279-serviceca\") pod \"node-ca-gnn6l\" (UID: \"c132d28e-79ff-4bef-8d61-b5a24f10a279\") " pod="openshift-image-registry/node-ca-gnn6l"
Jul 07 14:56:38.863304 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863180    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-system-cni-dir\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.863304 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863258    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-system-cni-dir\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.863304 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:38.863267    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:38.863304 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863294    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd8nw\" (UniqueName: \"kubernetes.io/projected/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-kube-api-access-bd8nw\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.863565 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863321    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-cni-binary-copy\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.863565 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:38.863367    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret podName:bde44302-557d-46b5-9d01-c43bcc78d2ec nodeName:}" failed. No retries permitted until 2026-07-07 14:56:39.363329247 +0000 UTC m=+3.036935290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret") pod "global-pull-secret-syncer-266nr" (UID: "bde44302-557d-46b5-9d01-c43bcc78d2ec") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:38.863565 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863447    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.863565 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863518    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-var-lib-kubelet\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.863565 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863536    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-multus-daemon-config\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.863565 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863554    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-cni-binary-copy\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863581    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-cnibin\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863597    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-var-lib-cni-multus\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863609    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-var-lib-kubelet\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863615    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/caee8dda-918b-405f-af0c-f7bb812ab885-hosts-file\") pod \"node-resolver-x9g98\" (UID: \"caee8dda-918b-405f-af0c-f7bb812ab885\") " pod="openshift-dns/node-resolver-x9g98"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863655    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/caee8dda-918b-405f-af0c-f7bb812ab885-hosts-file\") pod \"node-resolver-x9g98\" (UID: \"caee8dda-918b-405f-af0c-f7bb812ab885\") " pod="openshift-dns/node-resolver-x9g98"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863685    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c132d28e-79ff-4bef-8d61-b5a24f10a279-serviceca\") pod \"node-ca-gnn6l\" (UID: \"c132d28e-79ff-4bef-8d61-b5a24f10a279\") " pod="openshift-image-registry/node-ca-gnn6l"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863672    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c-agent-certs\") pod \"konnectivity-agent-cfssq\" (UID: \"3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c\") " pod="kube-system/konnectivity-agent-cfssq"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863726    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caee8dda-918b-405f-af0c-f7bb812ab885-tmp-dir\") pod \"node-resolver-x9g98\" (UID: \"caee8dda-918b-405f-af0c-f7bb812ab885\") " pod="openshift-dns/node-resolver-x9g98"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863757    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-cnibin\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfbv5\" (UniqueName: \"kubernetes.io/projected/caee8dda-918b-405f-af0c-f7bb812ab885-kube-api-access-cfbv5\") pod \"node-resolver-x9g98\" (UID: \"caee8dda-918b-405f-af0c-f7bb812ab885\") " pod="openshift-dns/node-resolver-x9g98"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863798    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863803    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863835    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-cni-binary-copy\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863852    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:38.863889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.863869    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-var-lib-cni-multus\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:38.863940    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:38.864001    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs podName:b0469d85-dc53-4bb1-891e-99ed1d55fee8 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:39.36398296 +0000 UTC m=+3.037588995 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs") pod "network-metrics-daemon-dzg7v" (UID: "b0469d85-dc53-4bb1-891e-99ed1d55fee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864032    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-multus-daemon-config\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864087    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c-konnectivity-ca\") pod \"konnectivity-agent-cfssq\" (UID: \"3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c\") " pod="kube-system/konnectivity-agent-cfssq"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864115    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-run-k8s-cni-cncf-io\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864147    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-cnibin\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864172    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-multus-conf-dir\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864253    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bde44302-557d-46b5-9d01-c43bcc78d2ec-kubelet-config\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864306    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bde44302-557d-46b5-9d01-c43bcc78d2ec-dbus\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864328    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caee8dda-918b-405f-af0c-f7bb812ab885-tmp-dir\") pod \"node-resolver-x9g98\" (UID: \"caee8dda-918b-405f-af0c-f7bb812ab885\") " pod="openshift-dns/node-resolver-x9g98"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864338    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-os-release\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864342    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-cnibin\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864364    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ljlz\" (UniqueName: \"kubernetes.io/projected/c132d28e-79ff-4bef-8d61-b5a24f10a279-kube-api-access-4ljlz\") pod \"node-ca-gnn6l\" (UID: \"c132d28e-79ff-4bef-8d61-b5a24f10a279\") " pod="openshift-image-registry/node-ca-gnn6l"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864392    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-multus-socket-dir-parent\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864394    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-run-k8s-cni-cncf-io\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.864675 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864443    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c132d28e-79ff-4bef-8d61-b5a24f10a279-host\") pod \"node-ca-gnn6l\" (UID: \"c132d28e-79ff-4bef-8d61-b5a24f10a279\") " pod="openshift-image-registry/node-ca-gnn6l"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864460    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bde44302-557d-46b5-9d01-c43bcc78d2ec-kubelet-config\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864468    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-os-release\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvdm4\" (UniqueName: \"kubernetes.io/projected/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-kube-api-access-jvdm4\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864505    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-multus-conf-dir\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864519    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-etc-kubernetes\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864569    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-hostroot\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864606    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-run-multus-certs\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864650    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-system-cni-dir\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864675    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-multus-cni-dir\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864715    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-var-lib-cni-bin\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864744    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wl8q\" (UniqueName: \"kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q\") pod \"network-check-target-7dqwt\" (UID: \"b8bd0203-d82d-4727-b612-ef4669b4eab3\") " pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864779    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtlkd\" (UniqueName: \"kubernetes.io/projected/b0469d85-dc53-4bb1-891e-99ed1d55fee8-kube-api-access-jtlkd\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864820    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-run-netns\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864904    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c-konnectivity-ca\") pod \"konnectivity-agent-cfssq\" (UID: \"3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c\") " pod="kube-system/konnectivity-agent-cfssq"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864946    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-run-netns\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864975    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-etc-kubernetes\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.864997    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-hostroot\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.865575 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.865041    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-run-multus-certs\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.866249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.865085    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-system-cni-dir\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.866249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.865099    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.866249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.865124    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bde44302-557d-46b5-9d01-c43bcc78d2ec-dbus\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:38.866249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.865100    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-cni-binary-copy\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.866249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.865153    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-multus-cni-dir\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.866249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.865178    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-os-release\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.866249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.865181    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-multus-socket-dir-parent\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.866249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.865243    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c132d28e-79ff-4bef-8d61-b5a24f10a279-host\") pod \"node-ca-gnn6l\" (UID: \"c132d28e-79ff-4bef-8d61-b5a24f10a279\") " pod="openshift-image-registry/node-ca-gnn6l"
Jul 07 14:56:38.866249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.865308    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-os-release\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.866249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.865353    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-host-var-lib-cni-bin\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.866249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.865378    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.866779 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.866759    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c-agent-certs\") pod \"konnectivity-agent-cfssq\" (UID: \"3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c\") " pod="kube-system/konnectivity-agent-cfssq"
Jul 07 14:56:38.886184 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.886103    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfbv5\" (UniqueName: \"kubernetes.io/projected/caee8dda-918b-405f-af0c-f7bb812ab885-kube-api-access-cfbv5\") pod \"node-resolver-x9g98\" (UID: \"caee8dda-918b-405f-af0c-f7bb812ab885\") " pod="openshift-dns/node-resolver-x9g98"
Jul 07 14:56:38.894154 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:38.894119    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:38.894154 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:38.894155    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:38.894351 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:38.894168    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5wl8q for pod openshift-network-diagnostics/network-check-target-7dqwt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:38.894351 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:38.894223    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q podName:b8bd0203-d82d-4727-b612-ef4669b4eab3 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:39.394205104 +0000 UTC m=+3.067811157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5wl8q" (UniqueName: "kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q") pod "network-check-target-7dqwt" (UID: "b8bd0203-d82d-4727-b612-ef4669b4eab3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:38.898779 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.898756    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtlkd\" (UniqueName: \"kubernetes.io/projected/b0469d85-dc53-4bb1-891e-99ed1d55fee8-kube-api-access-jtlkd\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:38.900921 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.900892    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd8nw\" (UniqueName: \"kubernetes.io/projected/8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a-kube-api-access-bd8nw\") pod \"multus-additional-cni-plugins-xdxzl\" (UID: \"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a\") " pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:38.901451 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.901430    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvdm4\" (UniqueName: \"kubernetes.io/projected/c23ce787-ca1c-4ee5-84b7-2b7aae623f51-kube-api-access-jvdm4\") pod \"multus-9k67d\" (UID: \"c23ce787-ca1c-4ee5-84b7-2b7aae623f51\") " pod="openshift-multus/multus-9k67d"
Jul 07 14:56:38.902909 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.902878    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ljlz\" (UniqueName: \"kubernetes.io/projected/c132d28e-79ff-4bef-8d61-b5a24f10a279-kube-api-access-4ljlz\") pod \"node-ca-gnn6l\" (UID: \"c132d28e-79ff-4bef-8d61-b5a24f10a279\") " pod="openshift-image-registry/node-ca-gnn6l"
Jul 07 14:56:38.956285 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.956252    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr"
Jul 07 14:56:38.967936 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.967911    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7s7fg"
Jul 07 14:56:38.981401 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.981370    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mbjwr"
Jul 07 14:56:38.987550 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:38.987530    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:56:39.004178 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:39.004149    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gnn6l"
Jul 07 14:56:39.014192 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:39.014170    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9k67d"
Jul 07 14:56:39.022789 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:39.022768    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x9g98"
Jul 07 14:56:39.030620 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:39.030594    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xdxzl"
Jul 07 14:56:39.036382 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:39.036365    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cfssq"
Jul 07 14:56:39.370274 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:39.370186    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:39.370274 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:39.370260    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:39.370487 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:39.370371    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:39.370487 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:39.370426    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret podName:bde44302-557d-46b5-9d01-c43bcc78d2ec nodeName:}" failed. No retries permitted until 2026-07-07 14:56:40.370410466 +0000 UTC m=+4.044016500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret") pod "global-pull-secret-syncer-266nr" (UID: "bde44302-557d-46b5-9d01-c43bcc78d2ec") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:39.370487 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:39.370372    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:39.370487 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:39.370478    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs podName:b0469d85-dc53-4bb1-891e-99ed1d55fee8 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:40.370466183 +0000 UTC m=+4.044072219 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs") pod "network-metrics-daemon-dzg7v" (UID: "b0469d85-dc53-4bb1-891e-99ed1d55fee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:39.471068 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:39.471026    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wl8q\" (UniqueName: \"kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q\") pod \"network-check-target-7dqwt\" (UID: \"b8bd0203-d82d-4727-b612-ef4669b4eab3\") " pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:39.471289 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:39.471266    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:39.471361 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:39.471298    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:39.471361 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:39.471314    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5wl8q for pod openshift-network-diagnostics/network-check-target-7dqwt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:39.471451 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:39.471384    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q podName:b8bd0203-d82d-4727-b612-ef4669b4eab3 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:40.471363371 +0000 UTC m=+4.144969411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5wl8q" (UniqueName: "kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q") pod "network-check-target-7dqwt" (UID: "b8bd0203-d82d-4727-b612-ef4669b4eab3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:39.707363 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:39.707260    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:51:39 +0000 UTC" deadline="2028-01-28 06:46:13.902287554 +0000 UTC"
Jul 07 14:56:39.707363 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:39.707315    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13671h49m34.194975824s"
Jul 07 14:56:39.787367 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:39.787340    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:39.787527 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:39.787441    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:56:40.027484 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:40.027299    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a7480b_096b_4bdf_8352_632d0216e013.slice/crio-02bd3a535d0e7b79192154de9e3c6b494bc9a6552fd3c2a7f986f002cef718ac WatchSource:0}: Error finding container 02bd3a535d0e7b79192154de9e3c6b494bc9a6552fd3c2a7f986f002cef718ac: Status 404 returned error can't find the container with id 02bd3a535d0e7b79192154de9e3c6b494bc9a6552fd3c2a7f986f002cef718ac
Jul 07 14:56:40.030102 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:40.030043    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98536497_3355_4ae8_84c4_cbf6a100056b.slice/crio-393703928e87d0b3fa57308f69da8bbc4f5f5d8712e6976614eb4912dfa2566d WatchSource:0}: Error finding container 393703928e87d0b3fa57308f69da8bbc4f5f5d8712e6976614eb4912dfa2566d: Status 404 returned error can't find the container with id 393703928e87d0b3fa57308f69da8bbc4f5f5d8712e6976614eb4912dfa2566d
Jul 07 14:56:40.033077 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:40.033054    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b8a0f26_98e2_4c8f_91db_4b6a4c6bf44c.slice/crio-1622528c1bbd5d43b7c146fcad8e5dd91864cccfc71676a2b35a5ec9aec3a6d7 WatchSource:0}: Error finding container 1622528c1bbd5d43b7c146fcad8e5dd91864cccfc71676a2b35a5ec9aec3a6d7: Status 404 returned error can't find the container with id 1622528c1bbd5d43b7c146fcad8e5dd91864cccfc71676a2b35a5ec9aec3a6d7
Jul 07 14:56:40.033498 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:40.033477    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb33fcf01_5654_4cff_a8df_b7294a3f712f.slice/crio-69917cbe3fbb5e7d1539d06f983e6643a73e605bf8ff1af8c27563ed9a3bf207 WatchSource:0}: Error finding container 69917cbe3fbb5e7d1539d06f983e6643a73e605bf8ff1af8c27563ed9a3bf207: Status 404 returned error can't find the container with id 69917cbe3fbb5e7d1539d06f983e6643a73e605bf8ff1af8c27563ed9a3bf207
Jul 07 14:56:40.035204 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:40.034745    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aecdfb7_b67c_4208_a8ce_6bd6e1ee2d0a.slice/crio-cfa2bf0455188846f886932346f996aa7847c8887f25cfe63ef2cd7520e2b922 WatchSource:0}: Error finding container cfa2bf0455188846f886932346f996aa7847c8887f25cfe63ef2cd7520e2b922: Status 404 returned error can't find the container with id cfa2bf0455188846f886932346f996aa7847c8887f25cfe63ef2cd7520e2b922
Jul 07 14:56:40.036546 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:40.036120    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc132d28e_79ff_4bef_8d61_b5a24f10a279.slice/crio-6e95b81e6aa3945f7fb92085b34fb3e008e55b16a3b3f32b3ebae520e2a46852 WatchSource:0}: Error finding container 6e95b81e6aa3945f7fb92085b34fb3e008e55b16a3b3f32b3ebae520e2a46852: Status 404 returned error can't find the container with id 6e95b81e6aa3945f7fb92085b34fb3e008e55b16a3b3f32b3ebae520e2a46852
Jul 07 14:56:40.038629 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:40.038578    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23ce787_ca1c_4ee5_84b7_2b7aae623f51.slice/crio-087fbc4f8fb057d80e4ee4f4c2917fdc0eb8273124d0257b51353d914cf3cc9c WatchSource:0}: Error finding container 087fbc4f8fb057d80e4ee4f4c2917fdc0eb8273124d0257b51353d914cf3cc9c: Status 404 returned error can't find the container with id 087fbc4f8fb057d80e4ee4f4c2917fdc0eb8273124d0257b51353d914cf3cc9c
Jul 07 14:56:40.039444 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:40.039312    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaee8dda_918b_405f_af0c_f7bb812ab885.slice/crio-11d489e8c04d1b1d4ffe951f67b2d020c1ca777fe6236a853d72993b9c65ba44 WatchSource:0}: Error finding container 11d489e8c04d1b1d4ffe951f67b2d020c1ca777fe6236a853d72993b9c65ba44: Status 404 returned error can't find the container with id 11d489e8c04d1b1d4ffe951f67b2d020c1ca777fe6236a853d72993b9c65ba44
Jul 07 14:56:40.065834 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:56:40.065812    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75ba721c_5d55_4692_ae73_5cd48df6d08c.slice/crio-01be5417125d9dcc93dedf280e4d30516f5c72e02570680865c20b2894f09a53 WatchSource:0}: Error finding container 01be5417125d9dcc93dedf280e4d30516f5c72e02570680865c20b2894f09a53: Status 404 returned error can't find the container with id 01be5417125d9dcc93dedf280e4d30516f5c72e02570680865c20b2894f09a53
Jul 07 14:56:40.378673 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.378480    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:40.378805 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.378708    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:40.378805 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:40.378651    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:40.378805 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:40.378791    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs podName:b0469d85-dc53-4bb1-891e-99ed1d55fee8 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:42.378776393 +0000 UTC m=+6.052382427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs") pod "network-metrics-daemon-dzg7v" (UID: "b0469d85-dc53-4bb1-891e-99ed1d55fee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:40.378929 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:40.378815    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:40.378929 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:40.378882    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret podName:bde44302-557d-46b5-9d01-c43bcc78d2ec nodeName:}" failed. No retries permitted until 2026-07-07 14:56:42.378870352 +0000 UTC m=+6.052476386 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret") pod "global-pull-secret-syncer-266nr" (UID: "bde44302-557d-46b5-9d01-c43bcc78d2ec") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:40.479878 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.479759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wl8q\" (UniqueName: \"kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q\") pod \"network-check-target-7dqwt\" (UID: \"b8bd0203-d82d-4727-b612-ef4669b4eab3\") " pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:40.480094 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:40.479924    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:40.480094 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:40.479959    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:40.480094 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:40.479978    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5wl8q for pod openshift-network-diagnostics/network-check-target-7dqwt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:40.480094 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:40.480047    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q podName:b8bd0203-d82d-4727-b612-ef4669b4eab3 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:42.480027039 +0000 UTC m=+6.153633088 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5wl8q" (UniqueName: "kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q") pod "network-check-target-7dqwt" (UID: "b8bd0203-d82d-4727-b612-ef4669b4eab3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:40.707913 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.707625    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:51:39 +0000 UTC" deadline="2027-12-01 03:51:39.361767621 +0000 UTC"
Jul 07 14:56:40.707913 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.707661    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12276h54m58.654111535s"
Jul 07 14:56:40.795444 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.795400    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:40.795632 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:40.795563    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:56:40.796054 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.796031    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:40.796190 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:40.796147    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:56:40.807620 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.807584    2582 generic.go:356] "Generic (PLEG): container finished" podID="2933e68aa89922301a1452efadf03267" containerID="2032862530c3afaf68a643ffcd4124ab1f72864abc231e8fb4d18671065e216b" exitCode=0
Jul 07 14:56:40.807768 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.807665    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal" event={"ID":"2933e68aa89922301a1452efadf03267","Type":"ContainerDied","Data":"2032862530c3afaf68a643ffcd4124ab1f72864abc231e8fb4d18671065e216b"}
Jul 07 14:56:40.828106 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.828002    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7s7fg" event={"ID":"75ba721c-5d55-4692-ae73-5cd48df6d08c","Type":"ContainerStarted","Data":"01be5417125d9dcc93dedf280e4d30516f5c72e02570680865c20b2894f09a53"}
Jul 07 14:56:40.835414 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.835382    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9k67d" event={"ID":"c23ce787-ca1c-4ee5-84b7-2b7aae623f51","Type":"ContainerStarted","Data":"087fbc4f8fb057d80e4ee4f4c2917fdc0eb8273124d0257b51353d914cf3cc9c"}
Jul 07 14:56:40.840577 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.840548    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x9g98" event={"ID":"caee8dda-918b-405f-af0c-f7bb812ab885","Type":"ContainerStarted","Data":"11d489e8c04d1b1d4ffe951f67b2d020c1ca777fe6236a853d72993b9c65ba44"}
Jul 07 14:56:40.855163 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.851505    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gnn6l" event={"ID":"c132d28e-79ff-4bef-8d61-b5a24f10a279","Type":"ContainerStarted","Data":"6e95b81e6aa3945f7fb92085b34fb3e008e55b16a3b3f32b3ebae520e2a46852"}
Jul 07 14:56:40.857434 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.857368    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xdxzl" event={"ID":"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a","Type":"ContainerStarted","Data":"cfa2bf0455188846f886932346f996aa7847c8887f25cfe63ef2cd7520e2b922"}
Jul 07 14:56:40.862761 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.862731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cfssq" event={"ID":"3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c","Type":"ContainerStarted","Data":"1622528c1bbd5d43b7c146fcad8e5dd91864cccfc71676a2b35a5ec9aec3a6d7"}
Jul 07 14:56:40.880322 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.880261    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-158.ec2.internal" event={"ID":"7649a49abd2efd166fe10d91dbf1337c","Type":"ContainerStarted","Data":"e62140981845b95b49c23214a2554d61d6ba6d91c90bcfda13f76425c3194556"}
Jul 07 14:56:40.895236 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.895187    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mbjwr" event={"ID":"b33fcf01-5654-4cff-a8df-b7294a3f712f","Type":"ContainerStarted","Data":"69917cbe3fbb5e7d1539d06f983e6643a73e605bf8ff1af8c27563ed9a3bf207"}
Jul 07 14:56:40.904904 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.904814    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr" event={"ID":"98536497-3355-4ae8-84c4-cbf6a100056b","Type":"ContainerStarted","Data":"393703928e87d0b3fa57308f69da8bbc4f5f5d8712e6976614eb4912dfa2566d"}
Jul 07 14:56:40.919952 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:40.919919    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb" event={"ID":"54a7480b-096b-4bdf-8352-632d0216e013","Type":"ContainerStarted","Data":"02bd3a535d0e7b79192154de9e3c6b494bc9a6552fd3c2a7f986f002cef718ac"}
Jul 07 14:56:41.788111 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:41.787483    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:41.788111 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:41.787629    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:56:41.960016 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:41.959937    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal" event={"ID":"2933e68aa89922301a1452efadf03267","Type":"ContainerStarted","Data":"015d52e0b5b45a2e264b8244916d45ade4d2d918a6246749bae2d3f89e6b47f7"}
Jul 07 14:56:41.984436 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:41.983443    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-158.ec2.internal" podStartSLOduration=3.983422077 podStartE2EDuration="3.983422077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:41.981645075 +0000 UTC m=+5.655251138" watchObservedRunningTime="2026-07-07 14:56:41.983422077 +0000 UTC m=+5.657028140"
Jul 07 14:56:41.984436 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:41.983915    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-158.ec2.internal" podStartSLOduration=3.983898634 podStartE2EDuration="3.983898634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:40.89834775 +0000 UTC m=+4.571953810" watchObservedRunningTime="2026-07-07 14:56:41.983898634 +0000 UTC m=+5.657504694"
Jul 07 14:56:42.397587 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:42.397549    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:42.397790 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:42.397615    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:42.397790 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:42.397774    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:42.397900 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:42.397840    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs podName:b0469d85-dc53-4bb1-891e-99ed1d55fee8 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:46.397820667 +0000 UTC m=+10.071426705 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs") pod "network-metrics-daemon-dzg7v" (UID: "b0469d85-dc53-4bb1-891e-99ed1d55fee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:42.397969 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:42.397932    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:42.398020 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:42.398003    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret podName:bde44302-557d-46b5-9d01-c43bcc78d2ec nodeName:}" failed. No retries permitted until 2026-07-07 14:56:46.397982333 +0000 UTC m=+10.071588374 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret") pod "global-pull-secret-syncer-266nr" (UID: "bde44302-557d-46b5-9d01-c43bcc78d2ec") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:42.498386 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:42.498342    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wl8q\" (UniqueName: \"kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q\") pod \"network-check-target-7dqwt\" (UID: \"b8bd0203-d82d-4727-b612-ef4669b4eab3\") " pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:42.498588 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:42.498558    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:42.498588 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:42.498583    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:42.498701 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:42.498597    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5wl8q for pod openshift-network-diagnostics/network-check-target-7dqwt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:42.498701 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:42.498666    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q podName:b8bd0203-d82d-4727-b612-ef4669b4eab3 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:46.498638809 +0000 UTC m=+10.172244848 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5wl8q" (UniqueName: "kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q") pod "network-check-target-7dqwt" (UID: "b8bd0203-d82d-4727-b612-ef4669b4eab3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:42.786981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:42.786728    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:42.786981 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:42.786886    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:56:42.790235 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:42.787317    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:42.790235 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:42.787426    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:56:43.787441 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:43.787399    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:43.787620 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:43.787549    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:56:44.787386 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:44.787344    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:44.787885 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:44.787500    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:56:44.788043 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:44.788021    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:44.788190 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:44.788164    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:56:45.788036 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:45.787524    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:45.788036 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:45.787665    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:56:46.431884 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:46.431278    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:46.431884 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:46.431367    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:46.431884 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:46.431517    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:46.431884 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:46.431572    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:46.431884 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:46.431577    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret podName:bde44302-557d-46b5-9d01-c43bcc78d2ec nodeName:}" failed. No retries permitted until 2026-07-07 14:56:54.431557956 +0000 UTC m=+18.105164009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret") pod "global-pull-secret-syncer-266nr" (UID: "bde44302-557d-46b5-9d01-c43bcc78d2ec") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:46.431884 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:46.431609    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs podName:b0469d85-dc53-4bb1-891e-99ed1d55fee8 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:54.431597804 +0000 UTC m=+18.105203845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs") pod "network-metrics-daemon-dzg7v" (UID: "b0469d85-dc53-4bb1-891e-99ed1d55fee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:46.532568 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:46.532486    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wl8q\" (UniqueName: \"kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q\") pod \"network-check-target-7dqwt\" (UID: \"b8bd0203-d82d-4727-b612-ef4669b4eab3\") " pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:46.532767 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:46.532657    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:46.532767 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:46.532686    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:46.532767 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:46.532701    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5wl8q for pod openshift-network-diagnostics/network-check-target-7dqwt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:46.532938 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:46.532773    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q podName:b8bd0203-d82d-4727-b612-ef4669b4eab3 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:54.532753134 +0000 UTC m=+18.206359187 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5wl8q" (UniqueName: "kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q") pod "network-check-target-7dqwt" (UID: "b8bd0203-d82d-4727-b612-ef4669b4eab3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:46.789108 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:46.788448    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:46.789108 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:46.788575    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:56:46.789108 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:46.788631    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:46.789108 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:46.788715    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:56:47.788084 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:47.787700    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:47.788084 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:47.787841    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:56:48.787797 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:48.787082    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:48.787797 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:48.787119    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:48.787797 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:48.787236    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:56:48.787797 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:48.787371    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:56:49.787232 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:49.787193    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:49.787414 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:49.787340    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:56:50.786911 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:50.786876    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:50.787381 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:50.787016    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:56:50.787381 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:50.787063    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:50.787381 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:50.787189    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:56:51.787284 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:51.787250    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:51.787804 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:51.787353    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:56:52.786879 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:52.786833    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:52.787077 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:52.786833    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:52.787077 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:52.787009    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:56:52.787204 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:52.787079    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:56:53.787638 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:53.787600    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:53.788182 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:53.787719    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:56:54.501629 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:54.501592    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:54.501820 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:54.501660    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:54.501820 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:54.501742    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:54.501820 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:54.501769    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:54.501820 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:54.501819    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret podName:bde44302-557d-46b5-9d01-c43bcc78d2ec nodeName:}" failed. No retries permitted until 2026-07-07 14:57:10.501800223 +0000 UTC m=+34.175406259 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret") pod "global-pull-secret-syncer-266nr" (UID: "bde44302-557d-46b5-9d01-c43bcc78d2ec") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:54.501988 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:54.501834    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs podName:b0469d85-dc53-4bb1-891e-99ed1d55fee8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:10.501827582 +0000 UTC m=+34.175433615 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs") pod "network-metrics-daemon-dzg7v" (UID: "b0469d85-dc53-4bb1-891e-99ed1d55fee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:54.602746 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:54.602703    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wl8q\" (UniqueName: \"kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q\") pod \"network-check-target-7dqwt\" (UID: \"b8bd0203-d82d-4727-b612-ef4669b4eab3\") " pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:54.602924 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:54.602892    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:54.602924 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:54.602915    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:54.603031 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:54.602927    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5wl8q for pod openshift-network-diagnostics/network-check-target-7dqwt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:54.603031 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:54.602993    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q podName:b8bd0203-d82d-4727-b612-ef4669b4eab3 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:10.602975221 +0000 UTC m=+34.276581268 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5wl8q" (UniqueName: "kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q") pod "network-check-target-7dqwt" (UID: "b8bd0203-d82d-4727-b612-ef4669b4eab3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:54.787665 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:54.787578    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:54.788092 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:54.787712    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:56:54.788092 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:54.787761    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:54.788092 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:54.787875    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:56:55.786870 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:55.786835    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:55.787059 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:55.786960    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:56:56.788147 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:56.788100    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:56.788611 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:56.788226    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:56.788611 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:56.788243    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:56:56.791106 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:56.789150    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:56:57.787663 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:57.787626    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:57.787849 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:57.787749    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:56:58.787081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:58.787038    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:56:58.787642 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:58.787038    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:56:58.787642 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:58.787216    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:56:58.787642 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:58.787254    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:56:59.787066 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:56:59.786645    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:56:59.787242 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:56:59.787216    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:57:00.002742 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.002708    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr" event={"ID":"98536497-3355-4ae8-84c4-cbf6a100056b","Type":"ContainerStarted","Data":"25d9e5f8425771e0d079382fd458054fc458eebee6ede1e4a9f826835bbedf5c"}
Jul 07 14:57:00.005603 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.005574    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb" event={"ID":"54a7480b-096b-4bdf-8352-632d0216e013","Type":"ContainerStarted","Data":"c28a5bb43dbf31165873f3e04be44072460055eb47c7e6cf5bcb5c03ac381155"}
Jul 07 14:57:00.005720 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.005662    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb" event={"ID":"54a7480b-096b-4bdf-8352-632d0216e013","Type":"ContainerStarted","Data":"4a64ee0f094175402c149c74437df4f6e8573115365350c5308832e48b816ded"}
Jul 07 14:57:00.005720 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.005709    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb" event={"ID":"54a7480b-096b-4bdf-8352-632d0216e013","Type":"ContainerStarted","Data":"28268153c27894b0985c8d67110907a4168e3712c75aa8b504da6060b38d1d8b"}
Jul 07 14:57:00.005823 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.005761    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb" event={"ID":"54a7480b-096b-4bdf-8352-632d0216e013","Type":"ContainerStarted","Data":"de5f53b931124ad90b349ee37963d0e6a2b4c63e2e1a13a2777ca31e3f10b580"}
Jul 07 14:57:00.005823 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.005805    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb" event={"ID":"54a7480b-096b-4bdf-8352-632d0216e013","Type":"ContainerStarted","Data":"67fe9635ba72af8fc7aa278ed21ff2fc8aea07ad5a6523949e3a3c345ee1c464"}
Jul 07 14:57:00.006977 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.006933    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7s7fg" event={"ID":"75ba721c-5d55-4692-ae73-5cd48df6d08c","Type":"ContainerStarted","Data":"03c3ced7b273e4ba031673cf0f412bb85cfd4c1264d1d318f28bcf9fd3b444b3"}
Jul 07 14:57:00.008384 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.008363    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9k67d" event={"ID":"c23ce787-ca1c-4ee5-84b7-2b7aae623f51","Type":"ContainerStarted","Data":"078d744eb7f1572bf980f2e28150c61ef6cd21f6040080fd59c98afe98614924"}
Jul 07 14:57:00.009872 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.009846    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x9g98" event={"ID":"caee8dda-918b-405f-af0c-f7bb812ab885","Type":"ContainerStarted","Data":"08855d74314c94dd3d6e00ec458cf2fb512d485debd7ddf9ffe6c747c603ffcc"}
Jul 07 14:57:00.011392 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.011369    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gnn6l" event={"ID":"c132d28e-79ff-4bef-8d61-b5a24f10a279","Type":"ContainerStarted","Data":"2d7174369e8ca545a90861bce8977d97393e9e1e577701499c3d7b65fd9e7627"}
Jul 07 14:57:00.013005 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.012978    2582 generic.go:356] "Generic (PLEG): container finished" podID="8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a" containerID="b49dc59cc5c59d1817cf887e3cbc43db55724d666ccac6dbc7b42e501c2635a2" exitCode=0
Jul 07 14:57:00.013093 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.013056    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xdxzl" event={"ID":"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a","Type":"ContainerDied","Data":"b49dc59cc5c59d1817cf887e3cbc43db55724d666ccac6dbc7b42e501c2635a2"}
Jul 07 14:57:00.014507 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.014424    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cfssq" event={"ID":"3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c","Type":"ContainerStarted","Data":"82228450d0120e2fd5d50d0ea41dae907b88ceef1969931218e77e464f48a382"}
Jul 07 14:57:00.047625 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.047575    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-x9g98" podStartSLOduration=3.604620799 podStartE2EDuration="23.04755916s" totalImagesPullingTime="19.442938361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:00.047316895 +0000 UTC m=+23.720922950" watchObservedRunningTime="2026-07-07 14:57:00.04755916 +0000 UTC m=+23.721165215"
Jul 07 14:57:00.047874 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.047851    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7s7fg" podStartSLOduration=3.594670674 podStartE2EDuration="23.047842773s" totalImagesPullingTime="19.453172099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:00.029202774 +0000 UTC m=+23.702808831" watchObservedRunningTime="2026-07-07 14:57:00.047842773 +0000 UTC m=+23.721448878"
Jul 07 14:57:00.068860 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.068811    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cfssq" podStartSLOduration=3.59640709 podStartE2EDuration="23.068794041s" totalImagesPullingTime="19.472386951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:00.068376878 +0000 UTC m=+23.741982934" watchObservedRunningTime="2026-07-07 14:57:00.068794041 +0000 UTC m=+23.742400095"
Jul 07 14:57:00.114837 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.114785    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gnn6l" podStartSLOduration=11.699428217 podStartE2EDuration="23.114769985s" totalImagesPullingTime="11.415341768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:00.08765489 +0000 UTC m=+23.761260944" watchObservedRunningTime="2026-07-07 14:57:00.114769985 +0000 UTC m=+23.788376106"
Jul 07 14:57:00.141482 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.141434    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9k67d" podStartSLOduration=3.644415695 podStartE2EDuration="23.141419061s" totalImagesPullingTime="19.497003366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:00.115390378 +0000 UTC m=+23.788996430" watchObservedRunningTime="2026-07-07 14:57:00.141419061 +0000 UTC m=+23.815025177"
Jul 07 14:57:00.787011 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.786947    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:57:00.787011 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:00.786981    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:57:00.787196 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:00.787066    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:57:00.787196 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:00.787158    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:57:01.018546 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:01.018159    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mbjwr" event={"ID":"b33fcf01-5654-4cff-a8df-b7294a3f712f","Type":"ContainerStarted","Data":"054c24390fddf179f9773d2a9ec92fdad5178ae407131a40d7e9283dac3c50da"}
Jul 07 14:57:01.021644 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:01.021607    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb" event={"ID":"54a7480b-096b-4bdf-8352-632d0216e013","Type":"ContainerStarted","Data":"0956b8e65ce0b35946bf97e083c86cd7a3ccc2368a685340e5b47306e564c030"}
Jul 07 14:57:01.036615 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:01.036566    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mbjwr" podStartSLOduration=4.566234859 podStartE2EDuration="24.036549262s" totalImagesPullingTime="19.470314403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:01.03638973 +0000 UTC m=+24.709995787" watchObservedRunningTime="2026-07-07 14:57:01.036549262 +0000 UTC m=+24.710155317"
Jul 07 14:57:01.190325 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:01.190297    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 14:57:01.726199 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:01.726068    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T14:57:01.190319612Z","UUID":"b77cf10f-3bcd-44c6-9c31-3e5ce36681b6","Handler":null,"Name":"","Endpoint":""}
Jul 07 14:57:01.727800 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:01.727771    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 14:57:01.727938 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:01.727819    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 14:57:01.787531 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:01.787487    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:01.787702 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:01.787612    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:57:02.026250 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:02.026157    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr" event={"ID":"98536497-3355-4ae8-84c4-cbf6a100056b","Type":"ContainerStarted","Data":"374ba870c8b2d42fcb2a1108fe1b9e94fef54c6bce27522676f1af88e5a0f862"}
Jul 07 14:57:02.624503 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:02.624278    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cfssq"
Jul 07 14:57:02.624871 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:02.624838    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cfssq"
Jul 07 14:57:02.786767 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:02.786731    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:57:02.787013 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:02.786862    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:57:02.787013 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:02.786964    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:57:02.787158 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:02.787095    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:57:03.031094 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:03.031050    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr" event={"ID":"98536497-3355-4ae8-84c4-cbf6a100056b","Type":"ContainerStarted","Data":"37f7f87e527accc3ab6d170409c23fe4dbc320523cc279d0f6cc657023f3f32d"}
Jul 07 14:57:03.034753 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:03.034728    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb" event={"ID":"54a7480b-096b-4bdf-8352-632d0216e013","Type":"ContainerStarted","Data":"5ab627587e67f6a934e1baeb730ac1da2a03bd2a65327c5621037b149b931051"}
Jul 07 14:57:03.034929 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:03.034903    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cfssq"
Jul 07 14:57:03.035384 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:03.035365    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cfssq"
Jul 07 14:57:03.055199 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:03.055156    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckqr" podStartSLOduration=3.909904171 podStartE2EDuration="26.055141398s" totalImagesPullingTime="22.145237227s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:03.053985571 +0000 UTC m=+26.727591628" watchObservedRunningTime="2026-07-07 14:57:03.055141398 +0000 UTC m=+26.728747444"
Jul 07 14:57:03.786649 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:03.786608    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:03.786928 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:03.786733    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:57:04.787405 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:04.787378    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:57:04.788049 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:04.787497    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:57:04.788049 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:04.787574    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:57:04.788049 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:04.787670    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:57:05.041429 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:05.041400    2582 generic.go:356] "Generic (PLEG): container finished" podID="8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a" containerID="ee97f997bd45fdd4255f3dcc03c008f43b82cc1fea4c2128f869efb22263b850" exitCode=0
Jul 07 14:57:05.041607 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:05.041496    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xdxzl" event={"ID":"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a","Type":"ContainerDied","Data":"ee97f997bd45fdd4255f3dcc03c008f43b82cc1fea4c2128f869efb22263b850"}
Jul 07 14:57:05.045209 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:05.045183    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb" event={"ID":"54a7480b-096b-4bdf-8352-632d0216e013","Type":"ContainerStarted","Data":"3ca47e685090a0c2f2572fd3aace8f68d785e5cc6cbd33e85944f167aff8e83e"}
Jul 07 14:57:05.097444 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:05.097380    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb" podStartSLOduration=8.55254058 podStartE2EDuration="28.097364417s" totalImagesPullingTime="19.544823837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:05.096983918 +0000 UTC m=+28.770589972" watchObservedRunningTime="2026-07-07 14:57:05.097364417 +0000 UTC m=+28.770970472"
Jul 07 14:57:05.786854 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:05.786823    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:05.787066 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:05.786921    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:57:06.047672 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:06.047596    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:57:06.048170 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:06.047681    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:57:06.048170 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:06.047731    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:57:06.062952 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:06.062926    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:57:06.063248 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:06.063229    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:57:06.526592 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:06.526359    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7dqwt"]
Jul 07 14:57:06.526843 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:06.526682    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:06.526843 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:06.526768    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:57:06.526941 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:06.526929    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-266nr"]
Jul 07 14:57:06.527037 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:06.527015    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:57:06.527168 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:06.527142    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:57:06.530007 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:06.527490    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dzg7v"]
Jul 07 14:57:06.530007 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:06.527952    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:57:06.530007 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:06.528092    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:57:07.051064 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:07.051031    2582 generic.go:356] "Generic (PLEG): container finished" podID="8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a" containerID="6b5aae963685d03176bfa47dacce46ac65e47467036d959cdb7312e744f515f2" exitCode=0
Jul 07 14:57:07.051470 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:07.051112    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xdxzl" event={"ID":"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a","Type":"ContainerDied","Data":"6b5aae963685d03176bfa47dacce46ac65e47467036d959cdb7312e744f515f2"}
Jul 07 14:57:07.787404 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:07.787374    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:57:07.787571 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:07.787379    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:07.787571 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:07.787487    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:57:07.787655 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:07.787584    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:57:07.787655 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:07.787379    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:57:07.787729 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:07.787679    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:57:09.058541 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:09.058503    2582 generic.go:356] "Generic (PLEG): container finished" podID="8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a" containerID="267f2e1d1aa7a6a3faa08849c30780d32da6ee94fb71b2e216ea11144cd2a943" exitCode=0
Jul 07 14:57:09.059116 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:09.058567    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xdxzl" event={"ID":"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a","Type":"ContainerDied","Data":"267f2e1d1aa7a6a3faa08849c30780d32da6ee94fb71b2e216ea11144cd2a943"}
Jul 07 14:57:09.255527 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:09.255482    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:09.390535 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:09.390462    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:09.486111 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:09.486081    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:09.786718 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:09.786678    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:09.786718 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:09.786697    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:57:09.786947 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:09.786685    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:57:09.786947 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:09.786799    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:57:09.786947 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:09.786896    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:57:09.787101 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:09.786990    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:57:10.525113 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:10.525065    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:57:10.525665 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:10.525156    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:57:10.525665 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:10.525227    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:57:10.525665 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:10.525243    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:57:10.525665 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:10.525298    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret podName:bde44302-557d-46b5-9d01-c43bcc78d2ec nodeName:}" failed. No retries permitted until 2026-07-07 14:57:42.52528285 +0000 UTC m=+66.198888884 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret") pod "global-pull-secret-syncer-266nr" (UID: "bde44302-557d-46b5-9d01-c43bcc78d2ec") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:57:10.525665 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:10.525318    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs podName:b0469d85-dc53-4bb1-891e-99ed1d55fee8 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:42.525309063 +0000 UTC m=+66.198915097 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs") pod "network-metrics-daemon-dzg7v" (UID: "b0469d85-dc53-4bb1-891e-99ed1d55fee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:57:10.625666 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:10.625627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wl8q\" (UniqueName: \"kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q\") pod \"network-check-target-7dqwt\" (UID: \"b8bd0203-d82d-4727-b612-ef4669b4eab3\") " pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:10.625838 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:10.625778    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:57:10.625838 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:10.625794    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:57:10.625838 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:10.625803    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5wl8q for pod openshift-network-diagnostics/network-check-target-7dqwt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:57:10.625987 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:10.625850    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q podName:b8bd0203-d82d-4727-b612-ef4669b4eab3 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:42.625838564 +0000 UTC m=+66.299444597 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5wl8q" (UniqueName: "kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q") pod "network-check-target-7dqwt" (UID: "b8bd0203-d82d-4727-b612-ef4669b4eab3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:57:11.786806 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:11.786768    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:11.787288 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:11.786877    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:57:11.787288 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:11.786878    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dqwt" podUID="b8bd0203-d82d-4727-b612-ef4669b4eab3"
Jul 07 14:57:11.787288 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:11.786887    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:57:11.787288 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:11.786998    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-266nr" podUID="bde44302-557d-46b5-9d01-c43bcc78d2ec"
Jul 07 14:57:11.787288 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:11.787099    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dzg7v" podUID="b0469d85-dc53-4bb1-891e-99ed1d55fee8"
Jul 07 14:57:12.499812 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:12.499768    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:12.557319 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:12.557240    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:12.739044 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:12.739005    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:13.187364 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.187336    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-158.ec2.internal" event="NodeReady"
Jul 07 14:57:13.187735 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.187470    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 14:57:13.245784 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.245629    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nvbd4"]
Jul 07 14:57:13.249506 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.249490    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rwg52"]
Jul 07 14:57:13.249642 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.249625    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.252289 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.252264    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rwg52"
Jul 07 14:57:13.254798 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.254781    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 14:57:13.254997 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.254985    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9hq6c\""
Jul 07 14:57:13.255301 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.255279    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 14:57:13.255398 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.255334    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 14:57:13.255398 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.255364    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 14:57:13.255499 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.255422    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 14:57:13.255876 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.255862    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tc5rk\""
Jul 07 14:57:13.266335 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.266284    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rwg52"]
Jul 07 14:57:13.267185 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.267169    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nvbd4"]
Jul 07 14:57:13.348139 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.348092    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54adbea3-fb47-41d6-8e45-1388111db0bb-config-volume\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.348291 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.348150    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.348291 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.348217    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9796\" (UniqueName: \"kubernetes.io/projected/54adbea3-fb47-41d6-8e45-1388111db0bb-kube-api-access-g9796\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.348291 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.348262    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54adbea3-fb47-41d6-8e45-1388111db0bb-tmp-dir\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.449595 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.449561    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxff2\" (UniqueName: \"kubernetes.io/projected/797b0a24-ae0e-4ff0-abd2-f6654de72287-kube-api-access-wxff2\") pod \"ingress-canary-rwg52\" (UID: \"797b0a24-ae0e-4ff0-abd2-f6654de72287\") " pod="openshift-ingress-canary/ingress-canary-rwg52"
Jul 07 14:57:13.449787 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.449607    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54adbea3-fb47-41d6-8e45-1388111db0bb-config-volume\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.449787 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.449666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.449787 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.449711    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9796\" (UniqueName: \"kubernetes.io/projected/54adbea3-fb47-41d6-8e45-1388111db0bb-kube-api-access-g9796\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.449787 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.449741    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/797b0a24-ae0e-4ff0-abd2-f6654de72287-cert\") pod \"ingress-canary-rwg52\" (UID: \"797b0a24-ae0e-4ff0-abd2-f6654de72287\") " pod="openshift-ingress-canary/ingress-canary-rwg52"
Jul 07 14:57:13.449787 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.449772    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54adbea3-fb47-41d6-8e45-1388111db0bb-tmp-dir\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.450048 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:13.449849    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:13.450048 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:13.449915    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls podName:54adbea3-fb47-41d6-8e45-1388111db0bb nodeName:}" failed. No retries permitted until 2026-07-07 14:57:13.949897407 +0000 UTC m=+37.623503463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls") pod "dns-default-nvbd4" (UID: "54adbea3-fb47-41d6-8e45-1388111db0bb") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:13.450048 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.449994    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54adbea3-fb47-41d6-8e45-1388111db0bb-tmp-dir\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.450294 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.450274    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54adbea3-fb47-41d6-8e45-1388111db0bb-config-volume\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.466293 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.466262    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9796\" (UniqueName: \"kubernetes.io/projected/54adbea3-fb47-41d6-8e45-1388111db0bb-kube-api-access-g9796\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.550428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.550340    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxff2\" (UniqueName: \"kubernetes.io/projected/797b0a24-ae0e-4ff0-abd2-f6654de72287-kube-api-access-wxff2\") pod \"ingress-canary-rwg52\" (UID: \"797b0a24-ae0e-4ff0-abd2-f6654de72287\") " pod="openshift-ingress-canary/ingress-canary-rwg52"
Jul 07 14:57:13.550428 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.550411    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/797b0a24-ae0e-4ff0-abd2-f6654de72287-cert\") pod \"ingress-canary-rwg52\" (UID: \"797b0a24-ae0e-4ff0-abd2-f6654de72287\") " pod="openshift-ingress-canary/ingress-canary-rwg52"
Jul 07 14:57:13.552837 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.552817    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/797b0a24-ae0e-4ff0-abd2-f6654de72287-cert\") pod \"ingress-canary-rwg52\" (UID: \"797b0a24-ae0e-4ff0-abd2-f6654de72287\") " pod="openshift-ingress-canary/ingress-canary-rwg52"
Jul 07 14:57:13.560467 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.560446    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxff2\" (UniqueName: \"kubernetes.io/projected/797b0a24-ae0e-4ff0-abd2-f6654de72287-kube-api-access-wxff2\") pod \"ingress-canary-rwg52\" (UID: \"797b0a24-ae0e-4ff0-abd2-f6654de72287\") " pod="openshift-ingress-canary/ingress-canary-rwg52"
Jul 07 14:57:13.580370 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.580333    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rwg52"
Jul 07 14:57:13.711047 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.711021    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rwg52"]
Jul 07 14:57:13.714162 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:57:13.714123    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod797b0a24_ae0e_4ff0_abd2_f6654de72287.slice/crio-bb8863b06dcf11f8a3a57879caaef7cfd8b194a1daefbecc1390dd31be6d5831 WatchSource:0}: Error finding container bb8863b06dcf11f8a3a57879caaef7cfd8b194a1daefbecc1390dd31be6d5831: Status 404 returned error can't find the container with id bb8863b06dcf11f8a3a57879caaef7cfd8b194a1daefbecc1390dd31be6d5831
Jul 07 14:57:13.787578 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.787542    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:57:13.787578 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.787580    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:13.787822 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.787542    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:57:13.790955 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.790928    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 14:57:13.790955 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.790954    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 14:57:13.791107 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.790981    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w68zb\""
Jul 07 14:57:13.791253 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.791238    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 14:57:13.791323 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.791264    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nm9h4\""
Jul 07 14:57:13.791323 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.791286    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 14:57:13.953599 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:13.953565    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:13.953767 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:13.953681    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:13.953767 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:13.953748    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls podName:54adbea3-fb47-41d6-8e45-1388111db0bb nodeName:}" failed. No retries permitted until 2026-07-07 14:57:14.953725836 +0000 UTC m=+38.627331869 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls") pod "dns-default-nvbd4" (UID: "54adbea3-fb47-41d6-8e45-1388111db0bb") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:14.071532 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:14.071439    2582 generic.go:356] "Generic (PLEG): container finished" podID="8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a" containerID="1c39cc751d7d8416a33ee153a964006896c6756bcc964ec490f0b6adb1ceee36" exitCode=0
Jul 07 14:57:14.071532 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:14.071517    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xdxzl" event={"ID":"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a","Type":"ContainerDied","Data":"1c39cc751d7d8416a33ee153a964006896c6756bcc964ec490f0b6adb1ceee36"}
Jul 07 14:57:14.072936 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:14.072647    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rwg52" event={"ID":"797b0a24-ae0e-4ff0-abd2-f6654de72287","Type":"ContainerStarted","Data":"bb8863b06dcf11f8a3a57879caaef7cfd8b194a1daefbecc1390dd31be6d5831"}
Jul 07 14:57:14.959775 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:14.959523    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:14.960186 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:14.959672    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:14.960186 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:14.959906    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls podName:54adbea3-fb47-41d6-8e45-1388111db0bb nodeName:}" failed. No retries permitted until 2026-07-07 14:57:16.959885887 +0000 UTC m=+40.633491934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls") pod "dns-default-nvbd4" (UID: "54adbea3-fb47-41d6-8e45-1388111db0bb") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:15.077474 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:15.077433    2582 generic.go:356] "Generic (PLEG): container finished" podID="8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a" containerID="7221e0862234d1fc9b88a035f8f95328089334ba7d11c3883799c99635456b35" exitCode=0
Jul 07 14:57:15.077624 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:15.077470    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xdxzl" event={"ID":"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a","Type":"ContainerDied","Data":"7221e0862234d1fc9b88a035f8f95328089334ba7d11c3883799c99635456b35"}
Jul 07 14:57:15.769332 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:15.769291    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:15.838555 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:15.838516    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:16.083778 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:16.083746    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xdxzl" event={"ID":"8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a","Type":"ContainerStarted","Data":"7b12ced4f2411eb5c3194e8de193e8fd0103bcad6e6c9546e906e5a8d023b6a1"}
Jul 07 14:57:16.115077 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:16.115019    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xdxzl" podStartSLOduration=8.937987485 podStartE2EDuration="39.114995681s" totalImagesPullingTime="30.177008196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:16.1143492 +0000 UTC m=+39.787955255" watchObservedRunningTime="2026-07-07 14:57:16.114995681 +0000 UTC m=+39.788601738"
Jul 07 14:57:16.247026 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:16.247001    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:16.346193 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:16.346103    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:16.871304 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:16.871267    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:16.971799 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:16.971762    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:16.971950 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:16.971905    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:16.972000 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:16.971962    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls podName:54adbea3-fb47-41d6-8e45-1388111db0bb nodeName:}" failed. No retries permitted until 2026-07-07 14:57:20.971948056 +0000 UTC m=+44.645554090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls") pod "dns-default-nvbd4" (UID: "54adbea3-fb47-41d6-8e45-1388111db0bb") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:16.972639 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:16.972618    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:17.087888 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:17.087842    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rwg52" event={"ID":"797b0a24-ae0e-4ff0-abd2-f6654de72287","Type":"ContainerStarted","Data":"338e4b615791a188f4d3be0612dd57b24a1ca8a7409b00e95d57525504b42efc"}
Jul 07 14:57:17.108077 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:17.108018    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rwg52" podStartSLOduration=1.605393373 podStartE2EDuration="4.108000875s" totalImagesPullingTime="2.502607502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:17.107924738 +0000 UTC m=+40.781530797" watchObservedRunningTime="2026-07-07 14:57:17.108000875 +0000 UTC m=+40.781606924"
Jul 07 14:57:20.999527 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:20.999478    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:20.999924 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:20.999624    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:20.999924 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:20.999699    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls podName:54adbea3-fb47-41d6-8e45-1388111db0bb nodeName:}" failed. No retries permitted until 2026-07-07 14:57:28.99968411 +0000 UTC m=+52.673290144 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls") pod "dns-default-nvbd4" (UID: "54adbea3-fb47-41d6-8e45-1388111db0bb") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:21.318229 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:21.318102    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:21.378079 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:21.378011    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:22.569404 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:22.569370    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:22.652364 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:22.652330    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:26.092932 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.092898    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"]
Jul 07 14:57:26.128446 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.128412    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"
Jul 07 14:57:26.141825 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.141801    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 07 14:57:26.142146 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.142116    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 07 14:57:26.146490 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.146468    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"]
Jul 07 14:57:26.148350 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.148331    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:57:26.157649 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.157628    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-r7q2c\""
Jul 07 14:57:26.158677 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.158663    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 07 14:57:26.194380 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.194350    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:26.236693 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.236662    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55c568f4c6-qgpts"]
Jul 07 14:57:26.236824 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.236731    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ce8379b2-ff5b-43f1-bd26-a89b83fc5732-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-vzst6\" (UID: \"ce8379b2-ff5b-43f1-bd26-a89b83fc5732\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"
Jul 07 14:57:26.236824 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.236757    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ce8379b2-ff5b-43f1-bd26-a89b83fc5732-config\") pod \"service-ca-operator-5d46d9ffd4-vzst6\" (UID: \"ce8379b2-ff5b-43f1-bd26-a89b83fc5732\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"
Jul 07 14:57:26.236824 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.236814    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52kwx\" (UniqueName: \"kubernetes.io/projected/ce8379b2-ff5b-43f1-bd26-a89b83fc5732-kube-api-access-52kwx\") pod \"service-ca-operator-5d46d9ffd4-vzst6\" (UID: \"ce8379b2-ff5b-43f1-bd26-a89b83fc5732\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"
Jul 07 14:57:26.260579 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.260553    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.263401 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.263376    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 14:57:26.263401 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.263392    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 14:57:26.263700 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.263669    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 14:57:26.263700 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.263690    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dszn9\""
Jul 07 14:57:26.267112 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.267060    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:26.267655 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.267635    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55c568f4c6-qgpts"]
Jul 07 14:57:26.269398 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.269376    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 14:57:26.337959 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.337926    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76752a32-bb77-4420-93ed-26cadf8e79e9-trusted-ca\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.338161 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.337977    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76752a32-bb77-4420-93ed-26cadf8e79e9-image-registry-private-configuration\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.338161 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.338004    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-bound-sa-token\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.338161 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.338043    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52kwx\" (UniqueName: \"kubernetes.io/projected/ce8379b2-ff5b-43f1-bd26-a89b83fc5732-kube-api-access-52kwx\") pod \"service-ca-operator-5d46d9ffd4-vzst6\" (UID: \"ce8379b2-ff5b-43f1-bd26-a89b83fc5732\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"
Jul 07 14:57:26.338161 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.338064    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.338161 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.338086    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76752a32-bb77-4420-93ed-26cadf8e79e9-installation-pull-secrets\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.338161 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.338108    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqpqt\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-kube-api-access-hqpqt\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.338374 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.338238    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76752a32-bb77-4420-93ed-26cadf8e79e9-ca-trust-extracted\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.338374 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.338294    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ce8379b2-ff5b-43f1-bd26-a89b83fc5732-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-vzst6\" (UID: \"ce8379b2-ff5b-43f1-bd26-a89b83fc5732\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"
Jul 07 14:57:26.338374 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.338316    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ce8379b2-ff5b-43f1-bd26-a89b83fc5732-config\") pod \"service-ca-operator-5d46d9ffd4-vzst6\" (UID: \"ce8379b2-ff5b-43f1-bd26-a89b83fc5732\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"
Jul 07 14:57:26.338374 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.338345    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-certificates\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.338808 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.338791    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ce8379b2-ff5b-43f1-bd26-a89b83fc5732-config\") pod \"service-ca-operator-5d46d9ffd4-vzst6\" (UID: \"ce8379b2-ff5b-43f1-bd26-a89b83fc5732\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"
Jul 07 14:57:26.341655 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.341633    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ce8379b2-ff5b-43f1-bd26-a89b83fc5732-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-vzst6\" (UID: \"ce8379b2-ff5b-43f1-bd26-a89b83fc5732\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"
Jul 07 14:57:26.347248 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.347197    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52kwx\" (UniqueName: \"kubernetes.io/projected/ce8379b2-ff5b-43f1-bd26-a89b83fc5732-kube-api-access-52kwx\") pod \"service-ca-operator-5d46d9ffd4-vzst6\" (UID: \"ce8379b2-ff5b-43f1-bd26-a89b83fc5732\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"
Jul 07 14:57:26.436649 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.436607    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"
Jul 07 14:57:26.438850 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.438825    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76752a32-bb77-4420-93ed-26cadf8e79e9-image-registry-private-configuration\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.438935 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.438859    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-bound-sa-token\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.438935 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.438879    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.438935 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.438894    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76752a32-bb77-4420-93ed-26cadf8e79e9-installation-pull-secrets\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.438935 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.438910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqpqt\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-kube-api-access-hqpqt\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.439122 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:26.438994    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:26.439122 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.438998    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76752a32-bb77-4420-93ed-26cadf8e79e9-ca-trust-extracted\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.439122 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.439091    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-certificates\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.439122 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:26.439013    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55c568f4c6-qgpts: secret "image-registry-tls" not found
Jul 07 14:57:26.439337 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.439162    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76752a32-bb77-4420-93ed-26cadf8e79e9-trusted-ca\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.439337 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:26.439208    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls podName:76752a32-bb77-4420-93ed-26cadf8e79e9 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:26.939184213 +0000 UTC m=+50.612790264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls") pod "image-registry-55c568f4c6-qgpts" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9") : secret "image-registry-tls" not found
Jul 07 14:57:26.439912 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.439879    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76752a32-bb77-4420-93ed-26cadf8e79e9-ca-trust-extracted\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.440082 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.440059    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-certificates\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.443103 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.440789    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76752a32-bb77-4420-93ed-26cadf8e79e9-trusted-ca\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.443851 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.443830    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76752a32-bb77-4420-93ed-26cadf8e79e9-image-registry-private-configuration\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.444820 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.444800    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76752a32-bb77-4420-93ed-26cadf8e79e9-installation-pull-secrets\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.451268 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.451239    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-bound-sa-token\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.451407 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.451392    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqpqt\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-kube-api-access-hqpqt\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.613447 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.613367    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6"]
Jul 07 14:57:26.616273 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:57:26.616245    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce8379b2_ff5b_43f1_bd26_a89b83fc5732.slice/crio-a444443e20414e6500967936c62a91f50a1416b8c34dcb65c092f1d86e5aa629 WatchSource:0}: Error finding container a444443e20414e6500967936c62a91f50a1416b8c34dcb65c092f1d86e5aa629: Status 404 returned error can't find the container with id a444443e20414e6500967936c62a91f50a1416b8c34dcb65c092f1d86e5aa629
Jul 07 14:57:26.944435 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:26.944343    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:26.944579 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:26.944502    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:26.944579 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:26.944517    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55c568f4c6-qgpts: secret "image-registry-tls" not found
Jul 07 14:57:26.944579 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:26.944573    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls podName:76752a32-bb77-4420-93ed-26cadf8e79e9 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:27.944554416 +0000 UTC m=+51.618160466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls") pod "image-registry-55c568f4c6-qgpts" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9") : secret "image-registry-tls" not found
Jul 07 14:57:27.110657 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:27.110623    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6" event={"ID":"ce8379b2-ff5b-43f1-bd26-a89b83fc5732","Type":"ContainerStarted","Data":"a444443e20414e6500967936c62a91f50a1416b8c34dcb65c092f1d86e5aa629"}
Jul 07 14:57:27.952172 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:27.952118    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:27.952334 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:27.952245    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:27.952334 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:27.952257    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55c568f4c6-qgpts: secret "image-registry-tls" not found
Jul 07 14:57:27.952334 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:27.952303    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls podName:76752a32-bb77-4420-93ed-26cadf8e79e9 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:29.952290695 +0000 UTC m=+53.625896728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls") pod "image-registry-55c568f4c6-qgpts" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9") : secret "image-registry-tls" not found
Jul 07 14:57:29.060349 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:29.060314    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:29.060749 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:29.060465    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:29.060749 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:29.060527    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls podName:54adbea3-fb47-41d6-8e45-1388111db0bb nodeName:}" failed. No retries permitted until 2026-07-07 14:57:45.060513125 +0000 UTC m=+68.734119157 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls") pod "dns-default-nvbd4" (UID: "54adbea3-fb47-41d6-8e45-1388111db0bb") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:29.566974 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:29.566937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:29.622299 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:29.622218    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:29.967161 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:29.967068    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:29.967284 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:29.967233    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:29.967284 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:29.967254    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55c568f4c6-qgpts: secret "image-registry-tls" not found
Jul 07 14:57:29.967360 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:29.967309    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls podName:76752a32-bb77-4420-93ed-26cadf8e79e9 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:33.967294257 +0000 UTC m=+57.640900289 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls") pod "image-registry-55c568f4c6-qgpts" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9") : secret "image-registry-tls" not found
Jul 07 14:57:29.992242 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:29.992213    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:29.999841 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:29.999809    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-xc6b9"]
Jul 07 14:57:30.022581 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.022558    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-xc6b9"]
Jul 07 14:57:30.022689 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.022668    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xc6b9"
Jul 07 14:57:30.025425 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.025405    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 07 14:57:30.025559 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.025544    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 07 14:57:30.025682 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.025665    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hthr2\""
Jul 07 14:57:30.068426 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.068392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwgfw\" (UniqueName: \"kubernetes.io/projected/956cfaa6-0356-40c3-a659-a7716aac6580-kube-api-access-fwgfw\") pod \"migrator-d57d58cc9-xc6b9\" (UID: \"956cfaa6-0356-40c3-a659-a7716aac6580\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xc6b9"
Jul 07 14:57:30.085611 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.085586    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:30.119951 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.119919    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6" event={"ID":"ce8379b2-ff5b-43f1-bd26-a89b83fc5732","Type":"ContainerStarted","Data":"295e08438f7ee12e2b8e38bd4f7887aa241079beba0b666bb9e98e0120bd400d"}
Jul 07 14:57:30.141981 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.141935    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6" podStartSLOduration=1.538322756 podStartE2EDuration="4.14192202s" totalImagesPullingTime="2.603599264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:30.141298602 +0000 UTC m=+53.814904657" watchObservedRunningTime="2026-07-07 14:57:30.14192202 +0000 UTC m=+53.815528074"
Jul 07 14:57:30.169863 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.169831    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwgfw\" (UniqueName: \"kubernetes.io/projected/956cfaa6-0356-40c3-a659-a7716aac6580-kube-api-access-fwgfw\") pod \"migrator-d57d58cc9-xc6b9\" (UID: \"956cfaa6-0356-40c3-a659-a7716aac6580\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xc6b9"
Jul 07 14:57:30.179334 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.179302    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwgfw\" (UniqueName: \"kubernetes.io/projected/956cfaa6-0356-40c3-a659-a7716aac6580-kube-api-access-fwgfw\") pod \"migrator-d57d58cc9-xc6b9\" (UID: \"956cfaa6-0356-40c3-a659-a7716aac6580\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xc6b9"
Jul 07 14:57:30.183727 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.183698    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:30.245044 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.244999    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t24sv"]
Jul 07 14:57:30.270353 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.270330    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t24sv"]
Jul 07 14:57:30.270469 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.270427    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.273366 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.273342    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 14:57:30.273478 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.273365    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 14:57:30.273478 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.273386    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 14:57:30.273478 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.273344    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c5zlx\""
Jul 07 14:57:30.273478 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.273438    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 14:57:30.294460 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.294435    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:30.331731 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.331706    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xc6b9"
Jul 07 14:57:30.372004 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.371970    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.372118 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.372073    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1cf8df9c-54d8-426d-9a67-4edfb2441335-data-volume\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.372203 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.372185    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.372279 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.372266    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1cf8df9c-54d8-426d-9a67-4edfb2441335-crio-socket\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.372324 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.372288    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nf8b\" (UniqueName: \"kubernetes.io/projected/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-api-access-9nf8b\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.377589 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.377566    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:30.472887 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.472858    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1cf8df9c-54d8-426d-9a67-4edfb2441335-crio-socket\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.472887 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.472892    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nf8b\" (UniqueName: \"kubernetes.io/projected/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-api-access-9nf8b\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.473114 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.472919    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.473114 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.472938    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1cf8df9c-54d8-426d-9a67-4edfb2441335-data-volume\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.473114 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.472996    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.473114 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:30.473052    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:30.473114 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:30.473114    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:30.473424 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.473118    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1cf8df9c-54d8-426d-9a67-4edfb2441335-crio-socket\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.473424 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:30.473120    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls podName:1cf8df9c-54d8-426d-9a67-4edfb2441335 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:30.973100312 +0000 UTC m=+54.646706361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t24sv" (UID: "1cf8df9c-54d8-426d-9a67-4edfb2441335") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:30.473424 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:30.473201    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm podName:1cf8df9c-54d8-426d-9a67-4edfb2441335 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:30.973185522 +0000 UTC m=+54.646791575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t24sv" (UID: "1cf8df9c-54d8-426d-9a67-4edfb2441335") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:30.473424 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.473365    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1cf8df9c-54d8-426d-9a67-4edfb2441335-data-volume\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.475019 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.474906    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-xc6b9"]
Jul 07 14:57:30.477539 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:57:30.477512    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod956cfaa6_0356_40c3_a659_a7716aac6580.slice/crio-0b168afa102e184436898ae7b74b0b1ae22c9e8ae9e3c3d34261a0df34692d9f WatchSource:0}: Error finding container 0b168afa102e184436898ae7b74b0b1ae22c9e8ae9e3c3d34261a0df34692d9f: Status 404 returned error can't find the container with id 0b168afa102e184436898ae7b74b0b1ae22c9e8ae9e3c3d34261a0df34692d9f
Jul 07 14:57:30.485220 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.485195    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nf8b\" (UniqueName: \"kubernetes.io/projected/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-api-access-9nf8b\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.933465 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.933420    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:30.976713 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.976682    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.976873 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:30.976735    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:30.976873 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:30.976798    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:30.976873 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:30.976825    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:30.976873 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:30.976848    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm podName:1cf8df9c-54d8-426d-9a67-4edfb2441335 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:31.976834611 +0000 UTC m=+55.650440644 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t24sv" (UID: "1cf8df9c-54d8-426d-9a67-4edfb2441335") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:30.977016 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:30.976877    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls podName:1cf8df9c-54d8-426d-9a67-4edfb2441335 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:31.976863488 +0000 UTC m=+55.650469525 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t24sv" (UID: "1cf8df9c-54d8-426d-9a67-4edfb2441335") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:31.029032 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:31.028999    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:31.123549 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:31.123513    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xc6b9" event={"ID":"956cfaa6-0356-40c3-a659-a7716aac6580","Type":"ContainerStarted","Data":"0b168afa102e184436898ae7b74b0b1ae22c9e8ae9e3c3d34261a0df34692d9f"}
Jul 07 14:57:31.382810 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:31.382774    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:31.445664 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:31.445572    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:31.626034 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:31.626001    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:31.708546 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:31.708464    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:31.984362 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:31.984328    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:31.984545 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:31.984417    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:31.984545 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:31.984471    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:31.984545 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:31.984540    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:31.984706 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:31.984544    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm podName:1cf8df9c-54d8-426d-9a67-4edfb2441335 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:33.984525213 +0000 UTC m=+57.658131247 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t24sv" (UID: "1cf8df9c-54d8-426d-9a67-4edfb2441335") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:31.984706 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:31.984606    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls podName:1cf8df9c-54d8-426d-9a67-4edfb2441335 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:33.984584063 +0000 UTC m=+57.658190100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t24sv" (UID: "1cf8df9c-54d8-426d-9a67-4edfb2441335") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:33.129394 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.129353    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xc6b9" event={"ID":"956cfaa6-0356-40c3-a659-a7716aac6580","Type":"ContainerStarted","Data":"2d892ba96a0df703e9bed534251eb6719313df6e244806de23b6c5000235c5a3"}
Jul 07 14:57:33.129874 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.129404    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xc6b9" event={"ID":"956cfaa6-0356-40c3-a659-a7716aac6580","Type":"ContainerStarted","Data":"a1c94993a415f5ced61f39c08600b31a863e184a7a56b9cb6bbc4e72fd01a95b"}
Jul 07 14:57:33.156305 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.156255    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xc6b9" podStartSLOduration=2.240630242 podStartE2EDuration="4.156239596s" totalImagesPullingTime="1.915609354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:33.155573641 +0000 UTC m=+56.829179696" watchObservedRunningTime="2026-07-07 14:57:33.156239596 +0000 UTC m=+56.829845694"
Jul 07 14:57:33.230184 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.230156    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x9g98_caee8dda-918b-405f-af0c-f7bb812ab885/dns-node-resolver/0.log"
Jul 07 14:57:33.310392 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.310362    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-njrxr"]
Jul 07 14:57:33.314178 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.313515    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-njrxr"
Jul 07 14:57:33.317046 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.317024    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 07 14:57:33.317182 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.317060    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9bnfn\""
Jul 07 14:57:33.317409 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.317394    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 07 14:57:33.317474 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.317459    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 07 14:57:33.317576 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.317555    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 07 14:57:33.322150 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.322115    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-njrxr"]
Jul 07 14:57:33.390962 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.390871    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:33.393970 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.393948    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5a5c6152-f70e-4594-b571-04b463e69dae-signing-key\") pod \"service-ca-758f599c5-njrxr\" (UID: \"5a5c6152-f70e-4594-b571-04b463e69dae\") " pod="openshift-service-ca/service-ca-758f599c5-njrxr"
Jul 07 14:57:33.394033 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.393988    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5a5c6152-f70e-4594-b571-04b463e69dae-signing-cabundle\") pod \"service-ca-758f599c5-njrxr\" (UID: \"5a5c6152-f70e-4594-b571-04b463e69dae\") " pod="openshift-service-ca/service-ca-758f599c5-njrxr"
Jul 07 14:57:33.394076 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.394061    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvkvr\" (UniqueName: \"kubernetes.io/projected/5a5c6152-f70e-4594-b571-04b463e69dae-kube-api-access-jvkvr\") pod \"service-ca-758f599c5-njrxr\" (UID: \"5a5c6152-f70e-4594-b571-04b463e69dae\") " pod="openshift-service-ca/service-ca-758f599c5-njrxr"
Jul 07 14:57:33.451058 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.451031    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:33.495174 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.495117    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvkvr\" (UniqueName: \"kubernetes.io/projected/5a5c6152-f70e-4594-b571-04b463e69dae-kube-api-access-jvkvr\") pod \"service-ca-758f599c5-njrxr\" (UID: \"5a5c6152-f70e-4594-b571-04b463e69dae\") " pod="openshift-service-ca/service-ca-758f599c5-njrxr"
Jul 07 14:57:33.495313 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.495217    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5a5c6152-f70e-4594-b571-04b463e69dae-signing-key\") pod \"service-ca-758f599c5-njrxr\" (UID: \"5a5c6152-f70e-4594-b571-04b463e69dae\") " pod="openshift-service-ca/service-ca-758f599c5-njrxr"
Jul 07 14:57:33.495313 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.495245    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5a5c6152-f70e-4594-b571-04b463e69dae-signing-cabundle\") pod \"service-ca-758f599c5-njrxr\" (UID: \"5a5c6152-f70e-4594-b571-04b463e69dae\") " pod="openshift-service-ca/service-ca-758f599c5-njrxr"
Jul 07 14:57:33.495849 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.495830    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5a5c6152-f70e-4594-b571-04b463e69dae-signing-cabundle\") pod \"service-ca-758f599c5-njrxr\" (UID: \"5a5c6152-f70e-4594-b571-04b463e69dae\") " pod="openshift-service-ca/service-ca-758f599c5-njrxr"
Jul 07 14:57:33.497628 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.497601    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5a5c6152-f70e-4594-b571-04b463e69dae-signing-key\") pod \"service-ca-758f599c5-njrxr\" (UID: \"5a5c6152-f70e-4594-b571-04b463e69dae\") " pod="openshift-service-ca/service-ca-758f599c5-njrxr"
Jul 07 14:57:33.503956 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.503938    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvkvr\" (UniqueName: \"kubernetes.io/projected/5a5c6152-f70e-4594-b571-04b463e69dae-kube-api-access-jvkvr\") pod \"service-ca-758f599c5-njrxr\" (UID: \"5a5c6152-f70e-4594-b571-04b463e69dae\") " pod="openshift-service-ca/service-ca-758f599c5-njrxr"
Jul 07 14:57:33.622757 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.622724    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-njrxr"
Jul 07 14:57:33.746745 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:33.746715    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-njrxr"]
Jul 07 14:57:33.749740 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:57:33.749709    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a5c6152_f70e_4594_b571_04b463e69dae.slice/crio-fdadd6929495a5c253af9b4c1d6a410caf1413a99f19e919252f5345a993ed34 WatchSource:0}: Error finding container fdadd6929495a5c253af9b4c1d6a410caf1413a99f19e919252f5345a993ed34: Status 404 returned error can't find the container with id fdadd6929495a5c253af9b4c1d6a410caf1413a99f19e919252f5345a993ed34
Jul 07 14:57:34.000169 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:34.000117    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:34.000353 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:34.000187    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:34.000353 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:34.000256    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:34.000353 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:34.000296    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:34.000353 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:34.000322    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55c568f4c6-qgpts: secret "image-registry-tls" not found
Jul 07 14:57:34.000353 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:34.000333    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:34.000530 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:34.000336    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:34.000530 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:34.000385    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls podName:76752a32-bb77-4420-93ed-26cadf8e79e9 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:42.000363892 +0000 UTC m=+65.673969971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls") pod "image-registry-55c568f4c6-qgpts" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9") : secret "image-registry-tls" not found
Jul 07 14:57:34.000530 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:34.000404    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls podName:1cf8df9c-54d8-426d-9a67-4edfb2441335 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:38.000395374 +0000 UTC m=+61.674001413 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t24sv" (UID: "1cf8df9c-54d8-426d-9a67-4edfb2441335") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:34.000530 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:34.000420    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm podName:1cf8df9c-54d8-426d-9a67-4edfb2441335 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:38.000411173 +0000 UTC m=+61.674017213 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t24sv" (UID: "1cf8df9c-54d8-426d-9a67-4edfb2441335") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:34.030873 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:34.030852    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gnn6l_c132d28e-79ff-4bef-8d61-b5a24f10a279/node-ca/0.log"
Jul 07 14:57:34.133877 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:34.133843    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-njrxr" event={"ID":"5a5c6152-f70e-4594-b571-04b463e69dae","Type":"ContainerStarted","Data":"d58cee1d504f13ed8ccb0187410267abbf425048843347ad96800fdb2b87814d"}
Jul 07 14:57:34.134280 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:34.133893    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-njrxr" event={"ID":"5a5c6152-f70e-4594-b571-04b463e69dae","Type":"ContainerStarted","Data":"fdadd6929495a5c253af9b4c1d6a410caf1413a99f19e919252f5345a993ed34"}
Jul 07 14:57:34.154046 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:34.154001    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-njrxr" podStartSLOduration=1.153983271 podStartE2EDuration="1.153983271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:34.152607558 +0000 UTC m=+57.826213616" watchObservedRunningTime="2026-07-07 14:57:34.153983271 +0000 UTC m=+57.827589352"
Jul 07 14:57:34.371389 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:34.371289    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:35.025937 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:35.025886    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rwg52_797b0a24-ae0e-4ff0-abd2-f6654de72287/serve-healthcheck-canary/0.log"
Jul 07 14:57:35.039695 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:35.039655    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:35.240239 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:35.240174    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-xc6b9_956cfaa6-0356-40c3-a659-a7716aac6580/migrator/0.log"
Jul 07 14:57:35.278074 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:35.277991    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:35.381082 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:35.381049    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:35.426567 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:35.426523    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-xc6b9_956cfaa6-0356-40c3-a659-a7716aac6580/graceful-termination/0.log"
Jul 07 14:57:35.467018 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:35.466979    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:35.571334 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:35.571239    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:35.783145 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:35.783107    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:36.625489 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:36.625448    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:36.731492 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:36.731460    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:37.202146 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:37.202102    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:37.344011 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:37.343969    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:37.530262 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:37.530230    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:37.962397 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:37.962312    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:38.032443 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:38.032408    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:38.032628 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:38.032528    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:38.032628 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:38.032532    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:38.032628 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:38.032615    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:38.032759 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:38.032646    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm podName:1cf8df9c-54d8-426d-9a67-4edfb2441335 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:46.032626845 +0000 UTC m=+69.706232882 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t24sv" (UID: "1cf8df9c-54d8-426d-9a67-4edfb2441335") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:38.032759 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:57:38.032674    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls podName:1cf8df9c-54d8-426d-9a67-4edfb2441335 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:46.032655829 +0000 UTC m=+69.706261862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t24sv" (UID: "1cf8df9c-54d8-426d-9a67-4edfb2441335") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:38.071229 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:38.071206    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9bhb"
Jul 07 14:57:38.528506 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:38.528473    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:38.763014 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:38.762982    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:38.965710 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:38.965625    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:39.043422 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:39.043376    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:39.339581 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:39.339549    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:39.561443 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:39.561409    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:39.619236 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:39.619144    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:40.135683 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:40.135647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:40.751330 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:40.751297    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:40.954485 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:40.954450    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:41.141355 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:41.141280    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:41.244973 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:41.244946    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:41.341823 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:41.341766    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:41.534412 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:41.534381    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:41.936391 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:41.936302    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:42.067276 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.067241    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:42.069569 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.069546    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls\") pod \"image-registry-55c568f4c6-qgpts\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") " pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:42.173524 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.173495    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dszn9\""
Jul 07 14:57:42.181928 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.181907    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:42.309662 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.309623    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55c568f4c6-qgpts"]
Jul 07 14:57:42.312966 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:57:42.312935    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76752a32_bb77_4420_93ed_26cadf8e79e9.slice/crio-eb42a14e5dc02be145821fef67b63e55c77b76fd3eeccd6b97778d209c22ae8b WatchSource:0}: Error finding container eb42a14e5dc02be145821fef67b63e55c77b76fd3eeccd6b97778d209c22ae8b: Status 404 returned error can't find the container with id eb42a14e5dc02be145821fef67b63e55c77b76fd3eeccd6b97778d209c22ae8b
Jul 07 14:57:42.530171 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.530119    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:42.571403 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.571367    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:57:42.571577 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.571423    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:57:42.574695 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.574664    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 14:57:42.574829 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.574772    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 14:57:42.584649 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.584629    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0469d85-dc53-4bb1-891e-99ed1d55fee8-metrics-certs\") pod \"network-metrics-daemon-dzg7v\" (UID: \"b0469d85-dc53-4bb1-891e-99ed1d55fee8\") " pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:57:42.584754 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.584650    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bde44302-557d-46b5-9d01-c43bcc78d2ec-original-pull-secret\") pod \"global-pull-secret-syncer-266nr\" (UID: \"bde44302-557d-46b5-9d01-c43bcc78d2ec\") " pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:57:42.597812 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.597782    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-266nr"
Jul 07 14:57:42.605356 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.605335    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nm9h4\""
Jul 07 14:57:42.612852 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.612830    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dzg7v"
Jul 07 14:57:42.673031 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.672512    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wl8q\" (UniqueName: \"kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q\") pod \"network-check-target-7dqwt\" (UID: \"b8bd0203-d82d-4727-b612-ef4669b4eab3\") " pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:42.676225 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.675972    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 14:57:42.686767 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.686716    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 14:57:42.696951 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.696916    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wl8q\" (UniqueName: \"kubernetes.io/projected/b8bd0203-d82d-4727-b612-ef4669b4eab3-kube-api-access-5wl8q\") pod \"network-check-target-7dqwt\" (UID: \"b8bd0203-d82d-4727-b612-ef4669b4eab3\") " pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:42.738742 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.738711    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-266nr"]
Jul 07 14:57:42.743581 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:57:42.743552    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde44302_557d_46b5_9d01_c43bcc78d2ec.slice/crio-7c26f98290036460fddb3ff7890f79ff49e121d250b74edf2e2d034f829be9d7 WatchSource:0}: Error finding container 7c26f98290036460fddb3ff7890f79ff49e121d250b74edf2e2d034f829be9d7: Status 404 returned error can't find the container with id 7c26f98290036460fddb3ff7890f79ff49e121d250b74edf2e2d034f829be9d7
Jul 07 14:57:42.765306 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.765273    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dzg7v"]
Jul 07 14:57:42.769516 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:57:42.769489    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0469d85_dc53_4bb1_891e_99ed1d55fee8.slice/crio-42090c8e8eb045011f56e5c7c765c4ecaa8a9fd86b3294c20d54e3f88bb3b410 WatchSource:0}: Error finding container 42090c8e8eb045011f56e5c7c765c4ecaa8a9fd86b3294c20d54e3f88bb3b410: Status 404 returned error can't find the container with id 42090c8e8eb045011f56e5c7c765c4ecaa8a9fd86b3294c20d54e3f88bb3b410
Jul 07 14:57:42.911139 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.911046    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w68zb\""
Jul 07 14:57:42.918231 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:42.918204    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:43.042220 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.042185    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7dqwt"]
Jul 07 14:57:43.045193 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:57:43.045168    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8bd0203_d82d_4727_b612_ef4669b4eab3.slice/crio-523050a0eac012b4333c9368162adfd972702ecf2ca2588f3fd441af9c5096b5 WatchSource:0}: Error finding container 523050a0eac012b4333c9368162adfd972702ecf2ca2588f3fd441af9c5096b5: Status 404 returned error can't find the container with id 523050a0eac012b4333c9368162adfd972702ecf2ca2588f3fd441af9c5096b5
Jul 07 14:57:43.139882 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.139849    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:43.157622 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.157585    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-266nr" event={"ID":"bde44302-557d-46b5-9d01-c43bcc78d2ec","Type":"ContainerStarted","Data":"7c26f98290036460fddb3ff7890f79ff49e121d250b74edf2e2d034f829be9d7"}
Jul 07 14:57:43.158645 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.158622    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7dqwt" event={"ID":"b8bd0203-d82d-4727-b612-ef4669b4eab3","Type":"ContainerStarted","Data":"523050a0eac012b4333c9368162adfd972702ecf2ca2588f3fd441af9c5096b5"}
Jul 07 14:57:43.159702 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.159682    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dzg7v" event={"ID":"b0469d85-dc53-4bb1-891e-99ed1d55fee8","Type":"ContainerStarted","Data":"42090c8e8eb045011f56e5c7c765c4ecaa8a9fd86b3294c20d54e3f88bb3b410"}
Jul 07 14:57:43.161062 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.161043    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55c568f4c6-qgpts" event={"ID":"76752a32-bb77-4420-93ed-26cadf8e79e9","Type":"ContainerStarted","Data":"8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9"}
Jul 07 14:57:43.161216 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.161078    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55c568f4c6-qgpts" event={"ID":"76752a32-bb77-4420-93ed-26cadf8e79e9","Type":"ContainerStarted","Data":"eb42a14e5dc02be145821fef67b63e55c77b76fd3eeccd6b97778d209c22ae8b"}
Jul 07 14:57:43.161216 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.161168    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:57:43.184493 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.184445    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-55c568f4c6-qgpts" podStartSLOduration=17.18443082 podStartE2EDuration="17.18443082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:43.184165035 +0000 UTC m=+66.857771088" watchObservedRunningTime="2026-07-07 14:57:43.18443082 +0000 UTC m=+66.858036874"
Jul 07 14:57:43.205463 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.205432    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:43.336288 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.336239    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:43.527698 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.527629    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:43.935997 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:43.935917    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:44.329155 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:44.329069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:44.762622 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:44.762574    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:45.093538 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:45.093497    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:45.096449 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:45.096416    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54adbea3-fb47-41d6-8e45-1388111db0bb-metrics-tls\") pod \"dns-default-nvbd4\" (UID: \"54adbea3-fb47-41d6-8e45-1388111db0bb\") " pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:45.175429 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:45.175392    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dzg7v" event={"ID":"b0469d85-dc53-4bb1-891e-99ed1d55fee8","Type":"ContainerStarted","Data":"cba8209f541ed0b30f90748e320ce6a6479552b7e6f82be390e667bea5ae5d93"}
Jul 07 14:57:45.175619 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:45.175464    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dzg7v" event={"ID":"b0469d85-dc53-4bb1-891e-99ed1d55fee8","Type":"ContainerStarted","Data":"79fb954d7166f4f7b8a136eca4265ea3158b5f767e81b7d779382d8063339708"}
Jul 07 14:57:45.194109 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:45.194051    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dzg7v" podStartSLOduration=66.556015814 podStartE2EDuration="1m8.194036024s" totalImagesPullingTime="1.63802021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:45.193630856 +0000 UTC m=+68.867236916" watchObservedRunningTime="2026-07-07 14:57:45.194036024 +0000 UTC m=+68.867642079"
Jul 07 14:57:45.338195 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:45.338161    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:45.379430 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:45.379357    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9hq6c\""
Jul 07 14:57:45.387202 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:45.387142    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:45.526579 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:45.526542    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:45.727673 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:45.727594    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:46.102693 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:46.102600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:46.102693 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:46.102687    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:46.103313 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:46.103286    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cf8df9c-54d8-426d-9a67-4edfb2441335-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:46.105076 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:46.105049    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cf8df9c-54d8-426d-9a67-4edfb2441335-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t24sv\" (UID: \"1cf8df9c-54d8-426d-9a67-4edfb2441335\") " pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:46.181674 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:46.181639    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c5zlx\""
Jul 07 14:57:46.189858 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:46.189834    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t24sv"
Jul 07 14:57:48.021308 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:48.021285    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t24sv"]
Jul 07 14:57:48.023722 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:57:48.023698    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf8df9c_54d8_426d_9a67_4edfb2441335.slice/crio-038899cc2820165dace73057369d27f4251fee3a88a6c5ab9064820a6c7d1e50 WatchSource:0}: Error finding container 038899cc2820165dace73057369d27f4251fee3a88a6c5ab9064820a6c7d1e50: Status 404 returned error can't find the container with id 038899cc2820165dace73057369d27f4251fee3a88a6c5ab9064820a6c7d1e50
Jul 07 14:57:48.037795 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:48.037767    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nvbd4"]
Jul 07 14:57:48.041222 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:57:48.041182    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54adbea3_fb47_41d6_8e45_1388111db0bb.slice/crio-df1f3b8176640f2cebdc995e8224003c04e2255c58b30505ab7e203f8ee9c560 WatchSource:0}: Error finding container df1f3b8176640f2cebdc995e8224003c04e2255c58b30505ab7e203f8ee9c560: Status 404 returned error can't find the container with id df1f3b8176640f2cebdc995e8224003c04e2255c58b30505ab7e203f8ee9c560
Jul 07 14:57:48.187889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:48.187794    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7dqwt" event={"ID":"b8bd0203-d82d-4727-b612-ef4669b4eab3","Type":"ContainerStarted","Data":"41037ce876f9a1073987ec61dfb503f712c94f26730c76e3016eaadd0aad3ecd"}
Jul 07 14:57:48.187889 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:48.187858    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:57:48.188999 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:48.188967    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t24sv" event={"ID":"1cf8df9c-54d8-426d-9a67-4edfb2441335","Type":"ContainerStarted","Data":"038899cc2820165dace73057369d27f4251fee3a88a6c5ab9064820a6c7d1e50"}
Jul 07 14:57:48.190241 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:48.190215    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-266nr" event={"ID":"bde44302-557d-46b5-9d01-c43bcc78d2ec","Type":"ContainerStarted","Data":"95d00379b658b051004930f2eed69b902c92606d89f47f9a1a235c63697f52af"}
Jul 07 14:57:48.191233 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:48.191208    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nvbd4" event={"ID":"54adbea3-fb47-41d6-8e45-1388111db0bb","Type":"ContainerStarted","Data":"df1f3b8176640f2cebdc995e8224003c04e2255c58b30505ab7e203f8ee9c560"}
Jul 07 14:57:48.204859 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:48.204813    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7dqwt" podStartSLOduration=66.371630432 podStartE2EDuration="1m11.204798916s" totalImagesPullingTime="4.833168484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:48.204119192 +0000 UTC m=+71.877725247" watchObservedRunningTime="2026-07-07 14:57:48.204798916 +0000 UTC m=+71.878404970"
Jul 07 14:57:48.218916 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:48.218867    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-266nr" podStartSLOduration=66.070635323 podStartE2EDuration="1m11.218847133s" totalImagesPullingTime="5.14821181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:48.218788498 +0000 UTC m=+71.892394554" watchObservedRunningTime="2026-07-07 14:57:48.218847133 +0000 UTC m=+71.892453189"
Jul 07 14:57:48.877438 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:48.877401    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:48.978580 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:48.978553    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:50.767419 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:50.767368    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:50.830055 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:50.830020    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:50.893943 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:50.893909    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:51.033852 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:51.033768    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:51.219491 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:51.219461    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:51.410990 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:51.410904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:51.464305 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:51.464271    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:52.208649 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:52.208613    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nvbd4" event={"ID":"54adbea3-fb47-41d6-8e45-1388111db0bb","Type":"ContainerStarted","Data":"a256ad8847aa056c56f5fe5e6f894b631179f905c26368246dde426844349d80"}
Jul 07 14:57:52.627642 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:52.627608    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:52.810474 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:52.810445    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:52.910991 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:52.910912    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:53.216960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:53.216857    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nvbd4" event={"ID":"54adbea3-fb47-41d6-8e45-1388111db0bb","Type":"ContainerStarted","Data":"ebb759fe97a69d4c245eb6e2af8c41186f486d30e796b9062d0fb57acdbc452b"}
Jul 07 14:57:53.216960 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:53.216927    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:57:53.218168 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:53.218146    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t24sv" event={"ID":"1cf8df9c-54d8-426d-9a67-4edfb2441335","Type":"ContainerStarted","Data":"3c432a417c55b3deda5fb4ccbc5eb3e1a6694d6001d6cb52ce7aa9d636fb1f5f"}
Jul 07 14:57:53.237740 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:53.237700    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nvbd4" podStartSLOduration=36.179274375 podStartE2EDuration="40.237686915s" totalImagesPullingTime="4.05841254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:53.235864169 +0000 UTC m=+76.909470424" watchObservedRunningTime="2026-07-07 14:57:53.237686915 +0000 UTC m=+76.911293014"
Jul 07 14:57:53.458389 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:53.458358    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:53.525301 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:53.525270    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:54.224543 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.224505    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t24sv" event={"ID":"1cf8df9c-54d8-426d-9a67-4edfb2441335","Type":"ContainerStarted","Data":"b3d83010fe26a9f8f1be9fa10edc8e12c795b75429c7f5ca420de208f6283fcb"}
Jul 07 14:57:54.287369 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.287330    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:54.365728 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.365690    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55c568f4c6-qgpts"]
Jul 07 14:57:54.406061 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.406023    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"]
Jul 07 14:57:54.409443 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.409421    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.413188 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.413167    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 07 14:57:54.418581 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.418296    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 07 14:57:54.418581 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.418311    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 14:57:54.418581 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.418363    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 14:57:54.423064 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.423044    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 07 14:57:54.423781 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.423765    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 14:57:54.424013 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.424000    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 07 14:57:54.438230 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.438173    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"]
Jul 07 14:57:54.440150 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.440106    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:54.466439 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.466404    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/00bfd191-1498-4b69-a287-04c80c6adc76-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.466594 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.466465    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbf8s\" (UniqueName: \"kubernetes.io/projected/00bfd191-1498-4b69-a287-04c80c6adc76-kube-api-access-cbf8s\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.466594 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.466564    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/00bfd191-1498-4b69-a287-04c80c6adc76-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.466712 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.466622    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/00bfd191-1498-4b69-a287-04c80c6adc76-hub\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.466712 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.466678    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/00bfd191-1498-4b69-a287-04c80c6adc76-ca\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.466712 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.466701    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00bfd191-1498-4b69-a287-04c80c6adc76-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.567249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.567166    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbf8s\" (UniqueName: \"kubernetes.io/projected/00bfd191-1498-4b69-a287-04c80c6adc76-kube-api-access-cbf8s\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.567249 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.567215    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/00bfd191-1498-4b69-a287-04c80c6adc76-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.567467 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.567251    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/00bfd191-1498-4b69-a287-04c80c6adc76-hub\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.567467 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.567305    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/00bfd191-1498-4b69-a287-04c80c6adc76-ca\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.567467 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.567331    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00bfd191-1498-4b69-a287-04c80c6adc76-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.567467 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.567376    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/00bfd191-1498-4b69-a287-04c80c6adc76-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.568279 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.568252    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/00bfd191-1498-4b69-a287-04c80c6adc76-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.570236 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.570211    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/00bfd191-1498-4b69-a287-04c80c6adc76-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.570382 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.570361    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/00bfd191-1498-4b69-a287-04c80c6adc76-hub\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.570662 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.570640    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/00bfd191-1498-4b69-a287-04c80c6adc76-ca\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.570998 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.570975    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00bfd191-1498-4b69-a287-04c80c6adc76-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.584782 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.584724    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbf8s\" (UniqueName: \"kubernetes.io/projected/00bfd191-1498-4b69-a287-04c80c6adc76-kube-api-access-cbf8s\") pod \"cluster-proxy-proxy-agent-6bd9cbc595-dhc6r\" (UID: \"00bfd191-1498-4b69-a287-04c80c6adc76\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.594972 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.594945    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:54.693195 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.693163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:54.730965 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.730922    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"
Jul 07 14:57:54.798929 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.798765    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:54.904985 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.904955    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r"]
Jul 07 14:57:54.907932 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:57:54.907896    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00bfd191_1498_4b69_a287_04c80c6adc76.slice/crio-72c28c3e9c8d9b060e490ff7361b594f7a32c6665a7de3e74210394ff2d4865e WatchSource:0}: Error finding container 72c28c3e9c8d9b060e490ff7361b594f7a32c6665a7de3e74210394ff2d4865e: Status 404 returned error can't find the container with id 72c28c3e9c8d9b060e490ff7361b594f7a32c6665a7de3e74210394ff2d4865e
Jul 07 14:57:54.915567 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:54.915537    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:55.229504 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:55.229467    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r" event={"ID":"00bfd191-1498-4b69-a287-04c80c6adc76","Type":"ContainerStarted","Data":"72c28c3e9c8d9b060e490ff7361b594f7a32c6665a7de3e74210394ff2d4865e"}
Jul 07 14:57:56.236069 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:56.236031    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t24sv" event={"ID":"1cf8df9c-54d8-426d-9a67-4edfb2441335","Type":"ContainerStarted","Data":"1a9237adbe2bd9edcef018047e1536ce69a51316a7e4a78d9be182e7f31f94e4"}
Jul 07 14:57:56.257511 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:56.257442    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t24sv" podStartSLOduration=18.308649366 podStartE2EDuration="26.257418054s" totalImagesPullingTime="7.948768688s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:57:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:56.256609247 +0000 UTC m=+79.930215306" watchObservedRunningTime="2026-07-07 14:57:56.257418054 +0000 UTC m=+79.931024110"
Jul 07 14:57:57.716375 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:57.716328    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:57.819864 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:57.819835    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:59.250051 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:59.250016    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r" event={"ID":"00bfd191-1498-4b69-a287-04c80c6adc76","Type":"ContainerStarted","Data":"fe1e740ac8e77c44c524eb8876cf22f3a21784243e21fcdfb8956aea2b50033b"}
Jul 07 14:57:59.250435 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:59.250062    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r" event={"ID":"00bfd191-1498-4b69-a287-04c80c6adc76","Type":"ContainerStarted","Data":"653d7e50bf4ef9a2ac258bd0c39013cd4945423a4d5e8a9a326ff7c6b6dc6fd0"}
Jul 07 14:57:59.250435 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:59.250074    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r" event={"ID":"00bfd191-1498-4b69-a287-04c80c6adc76","Type":"ContainerStarted","Data":"702581c5bbff8b85c6a07a93b09bff7a3e1c1b7780320573db361a644690bf32"}
Jul 07 14:57:59.273306 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:57:59.273256    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd9cbc595-dhc6r" podStartSLOduration=1.2312884689999999 podStartE2EDuration="5.273240091s" totalImagesPullingTime="4.041951622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:59.271707406 +0000 UTC m=+82.945313461" watchObservedRunningTime="2026-07-07 14:57:59.273240091 +0000 UTC m=+82.946846145"
Jul 07 14:58:01.225626 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:01.225590    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:01.395176 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:01.395124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:01.460485 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:01.460452    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:03.041555 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:03.041528    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:03.142623 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:03.142594    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:03.226507 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:03.226478    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nvbd4"
Jul 07 14:58:04.371010 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:04.370985    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:58:04.456995 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:04.456937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:04.565506 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:04.565474    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:04.873490 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:04.873455    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:04.973520 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:04.973488    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:07.054855 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:07.054807    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:07.136784 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:07.136749    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:08.151257 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:08.151228    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:08.247421 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:08.247392    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:10.853032 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:10.852997    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:10.916075 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:10.916047    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:13.238320 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:13.238284    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:13.341571 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:13.341532    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:13.960054 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:13.960024    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:15.281869 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:15.281837    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:16.652670 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:16.652642    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:16.718965 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:16.718936    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:17.240250 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.240221    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:17.340481 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.340447    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:17.634572 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.634447    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:17.750813 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.750772    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:17.858080 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.858042    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:17.940462 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.940377    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9fdsp"]
Jul 07 14:58:17.946765 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.946741    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:17.949433 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.949409    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 14:58:17.949584 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.949532    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 14:58:17.950697 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.950676    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fdkcq\""
Jul 07 14:58:17.950798 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.950725    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 14:58:17.950861 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.950808    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 14:58:17.950861 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.950841    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 14:58:17.950959 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.950937    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 14:58:17.963685 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:17.963662    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:18.035423 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.035392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6991c135-1f17-45fd-bdcd-7c94f4365586-sys\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.035604 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.035430    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-textfile\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.035604 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.035451    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6991c135-1f17-45fd-bdcd-7c94f4365586-metrics-client-ca\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.035604 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.035470    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.035604 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.035550    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6991c135-1f17-45fd-bdcd-7c94f4365586-root\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.035604 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.035592    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxcjf\" (UniqueName: \"kubernetes.io/projected/6991c135-1f17-45fd-bdcd-7c94f4365586-kube-api-access-wxcjf\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.035816 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.035631    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-wtmp\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.035816 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.035676    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-accelerators-collector-config\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.035816 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.035703    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-tls\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.038304 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.038282    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:18.131687 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.131660    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:18.136561 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.136541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-wtmp\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.136654 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.136575    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-accelerators-collector-config\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.136654 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.136600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-tls\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.136654 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.136632    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6991c135-1f17-45fd-bdcd-7c94f4365586-sys\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.136654 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.136653    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-textfile\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.136852 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.136676    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6991c135-1f17-45fd-bdcd-7c94f4365586-metrics-client-ca\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.136852 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.136705    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.136852 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.136721    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6991c135-1f17-45fd-bdcd-7c94f4365586-sys\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.136852 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.136731    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-wtmp\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.136852 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.136798    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6991c135-1f17-45fd-bdcd-7c94f4365586-root\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.136852 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.136830    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxcjf\" (UniqueName: \"kubernetes.io/projected/6991c135-1f17-45fd-bdcd-7c94f4365586-kube-api-access-wxcjf\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.137168 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.136916    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6991c135-1f17-45fd-bdcd-7c94f4365586-root\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.137168 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.137071    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-textfile\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.137290 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.137251    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-accelerators-collector-config\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.137362 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.137340    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6991c135-1f17-45fd-bdcd-7c94f4365586-metrics-client-ca\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.139087 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.139065    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.139180 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.139158    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6991c135-1f17-45fd-bdcd-7c94f4365586-node-exporter-tls\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.148290 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.148267    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxcjf\" (UniqueName: \"kubernetes.io/projected/6991c135-1f17-45fd-bdcd-7c94f4365586-kube-api-access-wxcjf\") pod \"node-exporter-9fdsp\" (UID: \"6991c135-1f17-45fd-bdcd-7c94f4365586\") " pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.231631 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.231597    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:18.256629 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.256601    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9fdsp"
Jul 07 14:58:18.270489 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:58:18.270466    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6991c135_1f17_45fd_bdcd_7c94f4365586.slice/crio-dd59ef0db3f87cbf5b0abeb4d11f3d72db75aa3fa1b9ecff4c366a73b22b18ff WatchSource:0}: Error finding container dd59ef0db3f87cbf5b0abeb4d11f3d72db75aa3fa1b9ecff4c366a73b22b18ff: Status 404 returned error can't find the container with id dd59ef0db3f87cbf5b0abeb4d11f3d72db75aa3fa1b9ecff4c366a73b22b18ff
Jul 07 14:58:18.312441 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.312411    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9fdsp" event={"ID":"6991c135-1f17-45fd-bdcd-7c94f4365586","Type":"ContainerStarted","Data":"dd59ef0db3f87cbf5b0abeb4d11f3d72db75aa3fa1b9ecff4c366a73b22b18ff"}
Jul 07 14:58:18.328867 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.328842    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:18.441410 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.441370    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:18.586028 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.585942    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:18.834700 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:18.834637    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:19.055533 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.055498    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:19.124679 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.124640    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:19.198846 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.198818    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7dqwt"
Jul 07 14:58:19.317933 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.317857    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9fdsp" event={"ID":"6991c135-1f17-45fd-bdcd-7c94f4365586","Type":"ContainerStarted","Data":"f29dc07391f0e055917735a66f7a0373cab5302f933c140174f8376f54d031e2"}
Jul 07 14:58:19.388605 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.388569    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-55c568f4c6-qgpts" podUID="76752a32-bb77-4420-93ed-26cadf8e79e9" containerName="registry" containerID="cri-o://8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9" gracePeriod=30
Jul 07 14:58:19.471519 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.471467    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:19.597355 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.597243    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:19.639855 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.639825    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:58:19.751150 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.751100    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqpqt\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-kube-api-access-hqpqt\") pod \"76752a32-bb77-4420-93ed-26cadf8e79e9\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") "
Jul 07 14:58:19.751322 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.751167    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-bound-sa-token\") pod \"76752a32-bb77-4420-93ed-26cadf8e79e9\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") "
Jul 07 14:58:19.751322 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.751198    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76752a32-bb77-4420-93ed-26cadf8e79e9-ca-trust-extracted\") pod \"76752a32-bb77-4420-93ed-26cadf8e79e9\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") "
Jul 07 14:58:19.751322 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.751231    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76752a32-bb77-4420-93ed-26cadf8e79e9-image-registry-private-configuration\") pod \"76752a32-bb77-4420-93ed-26cadf8e79e9\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") "
Jul 07 14:58:19.751322 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.751253    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76752a32-bb77-4420-93ed-26cadf8e79e9-trusted-ca\") pod \"76752a32-bb77-4420-93ed-26cadf8e79e9\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") "
Jul 07 14:58:19.751322 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.751273    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76752a32-bb77-4420-93ed-26cadf8e79e9-installation-pull-secrets\") pod \"76752a32-bb77-4420-93ed-26cadf8e79e9\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") "
Jul 07 14:58:19.751322 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.751301    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-certificates\") pod \"76752a32-bb77-4420-93ed-26cadf8e79e9\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") "
Jul 07 14:58:19.751576 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.751334    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls\") pod \"76752a32-bb77-4420-93ed-26cadf8e79e9\" (UID: \"76752a32-bb77-4420-93ed-26cadf8e79e9\") "
Jul 07 14:58:19.752002 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.751832    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76752a32-bb77-4420-93ed-26cadf8e79e9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "76752a32-bb77-4420-93ed-26cadf8e79e9" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:19.752123 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.752064    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "76752a32-bb77-4420-93ed-26cadf8e79e9" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:19.753882 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.753792    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "76752a32-bb77-4420-93ed-26cadf8e79e9" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:58:19.753882 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.753805    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76752a32-bb77-4420-93ed-26cadf8e79e9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "76752a32-bb77-4420-93ed-26cadf8e79e9" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:58:19.754061 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.753916    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-kube-api-access-hqpqt" (OuterVolumeSpecName: "kube-api-access-hqpqt") pod "76752a32-bb77-4420-93ed-26cadf8e79e9" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9"). InnerVolumeSpecName "kube-api-access-hqpqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:58:19.754061 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.753925    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76752a32-bb77-4420-93ed-26cadf8e79e9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "76752a32-bb77-4420-93ed-26cadf8e79e9" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:58:19.754061 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.753952    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "76752a32-bb77-4420-93ed-26cadf8e79e9" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:58:19.759759 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.759738    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76752a32-bb77-4420-93ed-26cadf8e79e9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "76752a32-bb77-4420-93ed-26cadf8e79e9" (UID: "76752a32-bb77-4420-93ed-26cadf8e79e9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:58:19.827740 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.827704    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:19.851846 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.851774    2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-tls\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.851846 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.851796    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqpqt\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-kube-api-access-hqpqt\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.851846 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.851806    2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76752a32-bb77-4420-93ed-26cadf8e79e9-bound-sa-token\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.851846 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.851815    2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76752a32-bb77-4420-93ed-26cadf8e79e9-ca-trust-extracted\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.851846 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.851824    2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76752a32-bb77-4420-93ed-26cadf8e79e9-image-registry-private-configuration\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.851846 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.851832    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76752a32-bb77-4420-93ed-26cadf8e79e9-trusted-ca\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.851846 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.851840    2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76752a32-bb77-4420-93ed-26cadf8e79e9-installation-pull-secrets\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.851846 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.851848    2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76752a32-bb77-4420-93ed-26cadf8e79e9-registry-certificates\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.939284 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:19.939249    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:20.322066 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.322032    2582 generic.go:356] "Generic (PLEG): container finished" podID="76752a32-bb77-4420-93ed-26cadf8e79e9" containerID="8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9" exitCode=0
Jul 07 14:58:20.322308 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.322084    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55c568f4c6-qgpts"
Jul 07 14:58:20.322308 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.322119    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55c568f4c6-qgpts" event={"ID":"76752a32-bb77-4420-93ed-26cadf8e79e9","Type":"ContainerDied","Data":"8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9"}
Jul 07 14:58:20.322308 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.322180    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55c568f4c6-qgpts" event={"ID":"76752a32-bb77-4420-93ed-26cadf8e79e9","Type":"ContainerDied","Data":"eb42a14e5dc02be145821fef67b63e55c77b76fd3eeccd6b97778d209c22ae8b"}
Jul 07 14:58:20.322308 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.322208    2582 scope.go:117] "RemoveContainer" containerID="8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9"
Jul 07 14:58:20.323659 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.323557    2582 generic.go:356] "Generic (PLEG): container finished" podID="6991c135-1f17-45fd-bdcd-7c94f4365586" containerID="f29dc07391f0e055917735a66f7a0373cab5302f933c140174f8376f54d031e2" exitCode=0
Jul 07 14:58:20.323659 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.323592    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9fdsp" event={"ID":"6991c135-1f17-45fd-bdcd-7c94f4365586","Type":"ContainerDied","Data":"f29dc07391f0e055917735a66f7a0373cab5302f933c140174f8376f54d031e2"}
Jul 07 14:58:20.340987 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.340970    2582 scope.go:117] "RemoveContainer" containerID="8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9"
Jul 07 14:58:20.341249 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:58:20.341230    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9\": container with ID starting with 8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9 not found: ID does not exist" containerID="8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9"
Jul 07 14:58:20.341334 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.341257    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9"} err="failed to get container status \"8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9\": rpc error: code = NotFound desc = could not find container \"8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9\": container with ID starting with 8ebaa986eb0b3e5c4351a431264a1e833c55d0876f6d589d1ddb4836eb304db9 not found: ID does not exist"
Jul 07 14:58:20.366928 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.366901    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55c568f4c6-qgpts"]
Jul 07 14:58:20.373387 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.373355    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-55c568f4c6-qgpts"]
Jul 07 14:58:20.791632 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.791555    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76752a32-bb77-4420-93ed-26cadf8e79e9" path="/var/lib/kubelet/pods/76752a32-bb77-4420-93ed-26cadf8e79e9/volumes"
Jul 07 14:58:20.859447 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.859412    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:20.955714 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:20.955683    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:21.250046 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:21.250009    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:21.329020 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:21.328986    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9fdsp" event={"ID":"6991c135-1f17-45fd-bdcd-7c94f4365586","Type":"ContainerStarted","Data":"4c16164b7416cac38a7d9519efe9469d73c07d980613a582f7febf8d3d1e403f"}
Jul 07 14:58:21.329240 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:21.329049    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9fdsp" event={"ID":"6991c135-1f17-45fd-bdcd-7c94f4365586","Type":"ContainerStarted","Data":"f45bc321baadd00c26ed65154553d77e139666ef31596eeab1dcfb486890c38c"}
Jul 07 14:58:21.354894 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:21.354845    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9fdsp" podStartSLOduration=3.351798686 podStartE2EDuration="4.354829636s" totalImagesPullingTime="938.607914ms" totalInitContainerRuntime="64.423036ms" isStatefulPod=true podCreationTimestamp="2026-07-07 14:58:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:21.3530507 +0000 UTC m=+105.026656769" watchObservedRunningTime="2026-07-07 14:58:21.354829636 +0000 UTC m=+105.028435690"
Jul 07 14:58:21.359596 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:21.359577    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:22.210641 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.210614    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:22.311917 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.311887    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:22.409094 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.409062    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:22.691711 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.691665    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"]
Jul 07 14:58:22.692031 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.692017    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76752a32-bb77-4420-93ed-26cadf8e79e9" containerName="registry"
Jul 07 14:58:22.692072 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.692035    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="76752a32-bb77-4420-93ed-26cadf8e79e9" containerName="registry"
Jul 07 14:58:22.692122 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.692113    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="76752a32-bb77-4420-93ed-26cadf8e79e9" containerName="registry"
Jul 07 14:58:22.740659 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.740385    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"]
Jul 07 14:58:22.740886 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.740749    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"
Jul 07 14:58:22.749996 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.749970    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2f25p\""
Jul 07 14:58:22.751319 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.751299    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 07 14:58:22.774932 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.774904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:22.874526 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.874490    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd1290ae-d56e-4d69-b315-e4a4346432a4-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-nlqbt\" (UID: \"fd1290ae-d56e-4d69-b315-e4a4346432a4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"
Jul 07 14:58:22.874697 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.874547    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89945\" (UniqueName: \"kubernetes.io/projected/fd1290ae-d56e-4d69-b315-e4a4346432a4-kube-api-access-89945\") pod \"monitoring-plugin-d5dcff7cf-nlqbt\" (UID: \"fd1290ae-d56e-4d69-b315-e4a4346432a4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"
Jul 07 14:58:22.927853 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.927824    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:22.975965 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.975933    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89945\" (UniqueName: \"kubernetes.io/projected/fd1290ae-d56e-4d69-b315-e4a4346432a4-kube-api-access-89945\") pod \"monitoring-plugin-d5dcff7cf-nlqbt\" (UID: \"fd1290ae-d56e-4d69-b315-e4a4346432a4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"
Jul 07 14:58:22.976163 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.976047    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd1290ae-d56e-4d69-b315-e4a4346432a4-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-nlqbt\" (UID: \"fd1290ae-d56e-4d69-b315-e4a4346432a4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"
Jul 07 14:58:22.978514 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.978492    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd1290ae-d56e-4d69-b315-e4a4346432a4-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-nlqbt\" (UID: \"fd1290ae-d56e-4d69-b315-e4a4346432a4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"
Jul 07 14:58:22.987608 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:22.987587    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89945\" (UniqueName: \"kubernetes.io/projected/fd1290ae-d56e-4d69-b315-e4a4346432a4-kube-api-access-89945\") pod \"monitoring-plugin-d5dcff7cf-nlqbt\" (UID: \"fd1290ae-d56e-4d69-b315-e4a4346432a4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"
Jul 07 14:58:23.016281 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:23.016243    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:23.053203 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:23.053174    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"
Jul 07 14:58:23.199861 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:23.199779    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"]
Jul 07 14:58:23.202565 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:58:23.202535    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1290ae_d56e_4d69_b315_e4a4346432a4.slice/crio-21554520be8d07f51368e3f8d0dc81e8bb04b88839b768705ed8840f0c6f4880 WatchSource:0}: Error finding container 21554520be8d07f51368e3f8d0dc81e8bb04b88839b768705ed8840f0c6f4880: Status 404 returned error can't find the container with id 21554520be8d07f51368e3f8d0dc81e8bb04b88839b768705ed8840f0c6f4880
Jul 07 14:58:23.338246 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:23.338168    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt" event={"ID":"fd1290ae-d56e-4d69-b315-e4a4346432a4","Type":"ContainerStarted","Data":"21554520be8d07f51368e3f8d0dc81e8bb04b88839b768705ed8840f0c6f4880"}
Jul 07 14:58:23.999007 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:23.998974    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67fb4b4b8f-mgtmh"]
Jul 07 14:58:24.016432 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.016402    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67fb4b4b8f-mgtmh"]
Jul 07 14:58:24.016580 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.016451    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.020468 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.020441    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 14:58:24.020764 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.020740    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dp7hd\""
Jul 07 14:58:24.021778 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.021757    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 14:58:24.022081 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.022065    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 14:58:24.022158 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.022088    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 14:58:24.022158 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.022066    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 14:58:24.022557 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.022539    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 14:58:24.023115 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.023092    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 14:58:24.026319 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.026300    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 14:58:24.183652 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.183613    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-trusted-ca-bundle\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.183820 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.183660    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxmj4\" (UniqueName: \"kubernetes.io/projected/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-kube-api-access-gxmj4\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.183820 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.183692    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-service-ca\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.183820 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.183718    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-serving-cert\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.183820 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.183742    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-oauth-serving-cert\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.183820 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.183808    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-oauth-config\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.183977 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.183853    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-config\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.256602 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.256507    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:24.284705 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.284669    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxmj4\" (UniqueName: \"kubernetes.io/projected/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-kube-api-access-gxmj4\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.284705 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.284714    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-service-ca\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.284955 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.284756    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-serving-cert\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.284955 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.284792    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-oauth-serving-cert\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.284955 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.284831    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-oauth-config\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.284955 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.284862    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-config\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.284955 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.284890    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-trusted-ca-bundle\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.285596 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.285572    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-service-ca\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.285806 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.285783    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-oauth-serving-cert\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.285958 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.285922    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-config\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.286076 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.286024    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-trusted-ca-bundle\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.287786 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.287767    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-serving-cert\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.287897 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.287824    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-oauth-config\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.300189 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.300162    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxmj4\" (UniqueName: \"kubernetes.io/projected/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-kube-api-access-gxmj4\") pod \"console-67fb4b4b8f-mgtmh\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") " pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.327871 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.327845    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:24.429528 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.429488    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:24.718269 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.718233    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67fb4b4b8f-mgtmh"]
Jul 07 14:58:24.740417 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.740379    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:24.846417 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:24.846368    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:24.847451 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:58:24.847430    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb72b7489_edc3_43a5_ba4c_2a1ce5126fda.slice/crio-f92092d9628cf2063267d54835f1bc402c336cd5f6d09b1337b44b0c70f049ea WatchSource:0}: Error finding container f92092d9628cf2063267d54835f1bc402c336cd5f6d09b1337b44b0c70f049ea: Status 404 returned error can't find the container with id f92092d9628cf2063267d54835f1bc402c336cd5f6d09b1337b44b0c70f049ea
Jul 07 14:58:25.347396 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:25.347366    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt" event={"ID":"fd1290ae-d56e-4d69-b315-e4a4346432a4","Type":"ContainerStarted","Data":"131d1330e5daaf84f006c67eaf096018c43b070006280f61f727bd30351ae61b"}
Jul 07 14:58:25.347612 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:25.347421    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"
Jul 07 14:58:25.348608 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:25.348580    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67fb4b4b8f-mgtmh" event={"ID":"b72b7489-edc3-43a5-ba4c-2a1ce5126fda","Type":"ContainerStarted","Data":"f92092d9628cf2063267d54835f1bc402c336cd5f6d09b1337b44b0c70f049ea"}
Jul 07 14:58:25.352365 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:25.352346    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt"
Jul 07 14:58:25.369352 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:25.369307    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nlqbt" podStartSLOduration=1.696801354 podStartE2EDuration="3.369293483s" totalImagesPullingTime="1.672492129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:25.367058166 +0000 UTC m=+109.040664221" watchObservedRunningTime="2026-07-07 14:58:25.369293483 +0000 UTC m=+109.042899538"
Jul 07 14:58:27.716238 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:27.716192    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67fb4b4b8f-mgtmh"]
Jul 07 14:58:28.361724 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:28.361691    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67fb4b4b8f-mgtmh" event={"ID":"b72b7489-edc3-43a5-ba4c-2a1ce5126fda","Type":"ContainerStarted","Data":"fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3"}
Jul 07 14:58:28.386577 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:28.386530    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67fb4b4b8f-mgtmh" podStartSLOduration=2.085744623 podStartE2EDuration="5.386513463s" totalImagesPullingTime="3.30076884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:28.385206029 +0000 UTC m=+112.058812086" watchObservedRunningTime="2026-07-07 14:58:28.386513463 +0000 UTC m=+112.060119519"
Jul 07 14:58:34.328169 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:34.328121    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:39.023617 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:39.023583    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:39.113682 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:39.113647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:39.225775 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:39.225731    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:39.333070 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:39.332990    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:45.669512 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:45.669474    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:45.768951 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:45.768918    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:45.869949 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:45.869881    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:46.234142 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:46.234094    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:46.298510 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:46.298473    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:46.423277 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:46.423248    2582 generic.go:356] "Generic (PLEG): container finished" podID="ce8379b2-ff5b-43f1-bd26-a89b83fc5732" containerID="295e08438f7ee12e2b8e38bd4f7887aa241079beba0b666bb9e98e0120bd400d" exitCode=0
Jul 07 14:58:46.423416 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:46.423319    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6" event={"ID":"ce8379b2-ff5b-43f1-bd26-a89b83fc5732","Type":"ContainerDied","Data":"295e08438f7ee12e2b8e38bd4f7887aa241079beba0b666bb9e98e0120bd400d"}
Jul 07 14:58:46.423634 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:46.423622    2582 scope.go:117] "RemoveContainer" containerID="295e08438f7ee12e2b8e38bd4f7887aa241079beba0b666bb9e98e0120bd400d"
Jul 07 14:58:47.428097 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:47.428065    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-vzst6" event={"ID":"ce8379b2-ff5b-43f1-bd26-a89b83fc5732","Type":"ContainerStarted","Data":"aa98878479c09875ffd14b71503158141f3cddca96558cf7cc5be7ac01d957ff"}
Jul 07 14:58:53.381492 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.381446    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-67fb4b4b8f-mgtmh" podUID="b72b7489-edc3-43a5-ba4c-2a1ce5126fda" containerName="console" containerID="cri-o://fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3" gracePeriod=15
Jul 07 14:58:53.622439 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.622414    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:53.683780 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.683679    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-config\") pod \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") "
Jul 07 14:58:53.683780 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.683736    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-serving-cert\") pod \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") "
Jul 07 14:58:53.683780 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.683767    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-oauth-serving-cert\") pod \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") "
Jul 07 14:58:53.684022 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.683792    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxmj4\" (UniqueName: \"kubernetes.io/projected/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-kube-api-access-gxmj4\") pod \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") "
Jul 07 14:58:53.684022 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.683827    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-service-ca\") pod \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") "
Jul 07 14:58:53.684022 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.683856    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-oauth-config\") pod \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") "
Jul 07 14:58:53.684022 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.683875    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-trusted-ca-bundle\") pod \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\" (UID: \"b72b7489-edc3-43a5-ba4c-2a1ce5126fda\") "
Jul 07 14:58:53.684339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.684306    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b72b7489-edc3-43a5-ba4c-2a1ce5126fda" (UID: "b72b7489-edc3-43a5-ba4c-2a1ce5126fda"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:53.684339 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.684324    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-config" (OuterVolumeSpecName: "console-config") pod "b72b7489-edc3-43a5-ba4c-2a1ce5126fda" (UID: "b72b7489-edc3-43a5-ba4c-2a1ce5126fda"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:53.684504 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.684361    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-service-ca" (OuterVolumeSpecName: "service-ca") pod "b72b7489-edc3-43a5-ba4c-2a1ce5126fda" (UID: "b72b7489-edc3-43a5-ba4c-2a1ce5126fda"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:53.684504 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.684461    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b72b7489-edc3-43a5-ba4c-2a1ce5126fda" (UID: "b72b7489-edc3-43a5-ba4c-2a1ce5126fda"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:53.686306 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.686267    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-kube-api-access-gxmj4" (OuterVolumeSpecName: "kube-api-access-gxmj4") pod "b72b7489-edc3-43a5-ba4c-2a1ce5126fda" (UID: "b72b7489-edc3-43a5-ba4c-2a1ce5126fda"). InnerVolumeSpecName "kube-api-access-gxmj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:58:53.686403 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.686302    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b72b7489-edc3-43a5-ba4c-2a1ce5126fda" (UID: "b72b7489-edc3-43a5-ba4c-2a1ce5126fda"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:58:53.686403 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.686350    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b72b7489-edc3-43a5-ba4c-2a1ce5126fda" (UID: "b72b7489-edc3-43a5-ba4c-2a1ce5126fda"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:58:53.784635 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.784596    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-config\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:53.784635 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.784625    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-serving-cert\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:53.784635 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.784635    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-oauth-serving-cert\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:53.784635 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.784644    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxmj4\" (UniqueName: \"kubernetes.io/projected/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-kube-api-access-gxmj4\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:53.784874 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.784653    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-service-ca\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:53.784874 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.784661    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-console-oauth-config\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:53.784874 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:53.784669    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b72b7489-edc3-43a5-ba4c-2a1ce5126fda-trusted-ca-bundle\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:54.455912 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:54.455872    2582 generic.go:356] "Generic (PLEG): container finished" podID="b72b7489-edc3-43a5-ba4c-2a1ce5126fda" containerID="fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3" exitCode=0
Jul 07 14:58:54.456321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:54.455933    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67fb4b4b8f-mgtmh"
Jul 07 14:58:54.456321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:54.455948    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67fb4b4b8f-mgtmh" event={"ID":"b72b7489-edc3-43a5-ba4c-2a1ce5126fda","Type":"ContainerDied","Data":"fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3"}
Jul 07 14:58:54.456321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:54.456022    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67fb4b4b8f-mgtmh" event={"ID":"b72b7489-edc3-43a5-ba4c-2a1ce5126fda","Type":"ContainerDied","Data":"f92092d9628cf2063267d54835f1bc402c336cd5f6d09b1337b44b0c70f049ea"}
Jul 07 14:58:54.456321 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:54.456041    2582 scope.go:117] "RemoveContainer" containerID="fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3"
Jul 07 14:58:54.464324 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:54.464307    2582 scope.go:117] "RemoveContainer" containerID="fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3"
Jul 07 14:58:54.464587 ip-10-0-140-158 kubenswrapper[2582]: E0707 14:58:54.464567    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3\": container with ID starting with fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3 not found: ID does not exist" containerID="fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3"
Jul 07 14:58:54.464634 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:54.464596    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3"} err="failed to get container status \"fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3\": rpc error: code = NotFound desc = could not find container \"fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3\": container with ID starting with fe4467602484aa7ddc4cc6ccb4926e62801edfcb9087527219325ddd5f8201e3 not found: ID does not exist"
Jul 07 14:58:54.477435 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:54.477407    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67fb4b4b8f-mgtmh"]
Jul 07 14:58:54.479859 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:54.479833    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67fb4b4b8f-mgtmh"]
Jul 07 14:58:54.791611 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:58:54.791577    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b72b7489-edc3-43a5-ba4c-2a1ce5126fda" path="/var/lib/kubelet/pods/b72b7489-edc3-43a5-ba4c-2a1ce5126fda/volumes"
Jul 07 14:59:34.127185 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.127101    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:34.197226 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.197194    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:34.841724 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.841686    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5788799c7-592fk"]
Jul 07 14:59:34.841947 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.841934    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72b7489-edc3-43a5-ba4c-2a1ce5126fda" containerName="console"
Jul 07 14:59:34.841989 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.841949    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72b7489-edc3-43a5-ba4c-2a1ce5126fda" containerName="console"
Jul 07 14:59:34.842026 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.841990    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b72b7489-edc3-43a5-ba4c-2a1ce5126fda" containerName="console"
Jul 07 14:59:34.844726 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.844702    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:34.847753 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.847735    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 14:59:34.848921 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.848896    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 14:59:34.849036 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.848931    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 14:59:34.849036 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.848958    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 14:59:34.849036 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.848960    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 14:59:34.852626 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.852605    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 14:59:34.852721 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.852638    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 14:59:34.852721 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.852613    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dp7hd\""
Jul 07 14:59:34.857254 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.857236    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 14:59:34.866312 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.866282    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5788799c7-592fk"]
Jul 07 14:59:34.894899 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.894869    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:34.971162 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.971116    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/353c01c7-cb71-4c54-bedd-8cec691bc245-console-oauth-config\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:34.971289 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.971172    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/353c01c7-cb71-4c54-bedd-8cec691bc245-console-config\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:34.971289 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.971233    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/353c01c7-cb71-4c54-bedd-8cec691bc245-service-ca\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:34.971289 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.971273    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/353c01c7-cb71-4c54-bedd-8cec691bc245-console-serving-cert\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:34.971424 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.971299    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9tqw\" (UniqueName: \"kubernetes.io/projected/353c01c7-cb71-4c54-bedd-8cec691bc245-kube-api-access-q9tqw\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:34.971424 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.971332    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/353c01c7-cb71-4c54-bedd-8cec691bc245-oauth-serving-cert\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:34.971424 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.971383    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353c01c7-cb71-4c54-bedd-8cec691bc245-trusted-ca-bundle\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:34.995758 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:34.995725    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:35.072751 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.072717    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/353c01c7-cb71-4c54-bedd-8cec691bc245-oauth-serving-cert\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.072905 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.072760    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353c01c7-cb71-4c54-bedd-8cec691bc245-trusted-ca-bundle\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.072905 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.072779    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/353c01c7-cb71-4c54-bedd-8cec691bc245-console-oauth-config\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.072905 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.072799    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/353c01c7-cb71-4c54-bedd-8cec691bc245-console-config\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.072905 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.072819    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/353c01c7-cb71-4c54-bedd-8cec691bc245-service-ca\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.072905 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.072848    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/353c01c7-cb71-4c54-bedd-8cec691bc245-console-serving-cert\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.072905 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.072876    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9tqw\" (UniqueName: \"kubernetes.io/projected/353c01c7-cb71-4c54-bedd-8cec691bc245-kube-api-access-q9tqw\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.073646 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.073613    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/353c01c7-cb71-4c54-bedd-8cec691bc245-oauth-serving-cert\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.073748 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.073684    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353c01c7-cb71-4c54-bedd-8cec691bc245-trusted-ca-bundle\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.073809 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.073769    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/353c01c7-cb71-4c54-bedd-8cec691bc245-console-config\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.073809 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.073793    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/353c01c7-cb71-4c54-bedd-8cec691bc245-service-ca\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.075873 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.075853    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/353c01c7-cb71-4c54-bedd-8cec691bc245-console-serving-cert\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.075987 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.075967    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/353c01c7-cb71-4c54-bedd-8cec691bc245-console-oauth-config\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.083502 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.083479    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9tqw\" (UniqueName: \"kubernetes.io/projected/353c01c7-cb71-4c54-bedd-8cec691bc245-kube-api-access-q9tqw\") pod \"console-5788799c7-592fk\" (UID: \"353c01c7-cb71-4c54-bedd-8cec691bc245\") " pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.153301 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.153209    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:35.282624 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.282604    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5788799c7-592fk"]
Jul 07 14:59:35.284399 ip-10-0-140-158 kubenswrapper[2582]: W0707 14:59:35.284378    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353c01c7_cb71_4c54_bedd_8cec691bc245.slice/crio-df55d3f19d14baecad35d9cc41ea9e0218d639e0b1d2945f1822c866a9cfa4a8 WatchSource:0}: Error finding container df55d3f19d14baecad35d9cc41ea9e0218d639e0b1d2945f1822c866a9cfa4a8: Status 404 returned error can't find the container with id df55d3f19d14baecad35d9cc41ea9e0218d639e0b1d2945f1822c866a9cfa4a8
Jul 07 14:59:35.594481 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.594441    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5788799c7-592fk" event={"ID":"353c01c7-cb71-4c54-bedd-8cec691bc245","Type":"ContainerStarted","Data":"bc56d311eb2bfd7e55871bdd3e96d42950eb3fc866bb1908fc17959117112f9f"}
Jul 07 14:59:35.594640 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.594503    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5788799c7-592fk" event={"ID":"353c01c7-cb71-4c54-bedd-8cec691bc245","Type":"ContainerStarted","Data":"df55d3f19d14baecad35d9cc41ea9e0218d639e0b1d2945f1822c866a9cfa4a8"}
Jul 07 14:59:35.618097 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:35.618051    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5788799c7-592fk" podStartSLOduration=1.618036911 podStartE2EDuration="1.618036911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:59:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:59:35.616264461 +0000 UTC m=+179.289870515" watchObservedRunningTime="2026-07-07 14:59:35.618036911 +0000 UTC m=+179.291642968"
Jul 07 14:59:45.154010 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:45.153972    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:45.154869 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:45.154041    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:45.158754 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:45.158730    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:45.632674 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:45.632645    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5788799c7-592fk"
Jul 07 14:59:45.797376 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:45.797347    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:45.898150 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:45.898045    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:45.955463 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:45.955431    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:46.174767 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:46.174690    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:46.236920 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:46.236886    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:54.237554 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:54.237522    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:54.340439 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:54.340408    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:54.407425 ip-10-0-140-158 kubenswrapper[2582]: I0707 14:59:54.407392    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:32.267621 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:00:32.267587    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:32.334242 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:00:32.334211    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:35.237576 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:00:35.237538    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:01.326885 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:01.326778    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:01.420602 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:01.420548    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:01.779018 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:01.778979    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:01.881160 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:01.881066    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:01.982800 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:01.982771    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:26.284031 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:26.283989    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:30.044166 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:30.044099    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:30.147561 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:30.147524    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:35.574090 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:35.574052    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:35.935614 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:35.935520    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:36.114143 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.114106    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:36.660586 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.660559    2582 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 15:01:36.701524 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.701491    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:36.936691 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.936488    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"]
Jul 07 15:01:36.940700 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.939597    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:01:36.944299 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.944275    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 07 15:01:36.944428 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.944275    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 07 15:01:36.944961 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.944943    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-gfqdh\""
Jul 07 15:01:36.945107 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.945092    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 07 15:01:36.945517 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.945496    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 07 15:01:36.945606 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.945538    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 07 15:01:36.946343 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.946320    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:36.959825 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:36.959798    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"]
Jul 07 15:01:37.024036 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.023996    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-d8znq\" (UID: \"3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:01:37.024036 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.024036    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3-certificates\") pod \"keda-metrics-apiserver-559f75f947-d8znq\" (UID: \"3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:01:37.024274 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.024171    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9952\" (UniqueName: \"kubernetes.io/projected/3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3-kube-api-access-j9952\") pod \"keda-metrics-apiserver-559f75f947-d8znq\" (UID: \"3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:01:37.124817 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.124778    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9952\" (UniqueName: \"kubernetes.io/projected/3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3-kube-api-access-j9952\") pod \"keda-metrics-apiserver-559f75f947-d8znq\" (UID: \"3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:01:37.125013 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.124834    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-d8znq\" (UID: \"3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:01:37.125013 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.124864    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3-certificates\") pod \"keda-metrics-apiserver-559f75f947-d8znq\" (UID: \"3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:01:37.125257 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.125238    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-d8znq\" (UID: \"3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:01:37.127446 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.127420    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3-certificates\") pod \"keda-metrics-apiserver-559f75f947-d8znq\" (UID: \"3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:01:37.135520 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.135497    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9952\" (UniqueName: \"kubernetes.io/projected/3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3-kube-api-access-j9952\") pod \"keda-metrics-apiserver-559f75f947-d8znq\" (UID: \"3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:01:37.143109 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.143080    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:37.157325 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.157298    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-mmcxh"]
Jul 07 15:01:37.160370 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.160351    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh"
Jul 07 15:01:37.163667 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.163637    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 07 15:01:37.172517 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.172495    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-mmcxh"]
Jul 07 15:01:37.225570 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.225535    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/077ffde3-d76a-472a-b43c-8c3a7c8f9e0e-certificates\") pod \"keda-admission-5fbd5c4644-mmcxh\" (UID: \"077ffde3-d76a-472a-b43c-8c3a7c8f9e0e\") " pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh"
Jul 07 15:01:37.225725 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.225599    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6kn8\" (UniqueName: \"kubernetes.io/projected/077ffde3-d76a-472a-b43c-8c3a7c8f9e0e-kube-api-access-b6kn8\") pod \"keda-admission-5fbd5c4644-mmcxh\" (UID: \"077ffde3-d76a-472a-b43c-8c3a7c8f9e0e\") " pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh"
Jul 07 15:01:37.240511 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.240485    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:37.249503 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.249475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:01:37.326575 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.326542    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/077ffde3-d76a-472a-b43c-8c3a7c8f9e0e-certificates\") pod \"keda-admission-5fbd5c4644-mmcxh\" (UID: \"077ffde3-d76a-472a-b43c-8c3a7c8f9e0e\") " pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh"
Jul 07 15:01:37.326743 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.326604    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6kn8\" (UniqueName: \"kubernetes.io/projected/077ffde3-d76a-472a-b43c-8c3a7c8f9e0e-kube-api-access-b6kn8\") pod \"keda-admission-5fbd5c4644-mmcxh\" (UID: \"077ffde3-d76a-472a-b43c-8c3a7c8f9e0e\") " pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh"
Jul 07 15:01:37.326926 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:01:37.326899    2582 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Jul 07 15:01:37.327048 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:01:37.326934    2582 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-5fbd5c4644-mmcxh: secret "keda-admission-webhooks-certs" not found
Jul 07 15:01:37.327048 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:01:37.326996    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/077ffde3-d76a-472a-b43c-8c3a7c8f9e0e-certificates podName:077ffde3-d76a-472a-b43c-8c3a7c8f9e0e nodeName:}" failed. No retries permitted until 2026-07-07 15:01:37.826977527 +0000 UTC m=+301.500583574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/077ffde3-d76a-472a-b43c-8c3a7c8f9e0e-certificates") pod "keda-admission-5fbd5c4644-mmcxh" (UID: "077ffde3-d76a-472a-b43c-8c3a7c8f9e0e") : secret "keda-admission-webhooks-certs" not found
Jul 07 15:01:37.337077 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.337044    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6kn8\" (UniqueName: \"kubernetes.io/projected/077ffde3-d76a-472a-b43c-8c3a7c8f9e0e-kube-api-access-b6kn8\") pod \"keda-admission-5fbd5c4644-mmcxh\" (UID: \"077ffde3-d76a-472a-b43c-8c3a7c8f9e0e\") " pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh"
Jul 07 15:01:37.395780 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.395740    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"]
Jul 07 15:01:37.399661 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:01:37.399623    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c2e9f71_4282_4ac1_9ba4_3a059d3c66c3.slice/crio-63fd561393b51d8f05370c5527a040e8f371f2898162f1fadeee10ae8631d136 WatchSource:0}: Error finding container 63fd561393b51d8f05370c5527a040e8f371f2898162f1fadeee10ae8631d136: Status 404 returned error can't find the container with id 63fd561393b51d8f05370c5527a040e8f371f2898162f1fadeee10ae8631d136
Jul 07 15:01:37.519622 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.519533    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:37.625015 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.624982    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:37.716745 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.716713    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:37.831299 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.831213    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/077ffde3-d76a-472a-b43c-8c3a7c8f9e0e-certificates\") pod \"keda-admission-5fbd5c4644-mmcxh\" (UID: \"077ffde3-d76a-472a-b43c-8c3a7c8f9e0e\") " pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh"
Jul 07 15:01:37.833923 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.833892    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/077ffde3-d76a-472a-b43c-8c3a7c8f9e0e-certificates\") pod \"keda-admission-5fbd5c4644-mmcxh\" (UID: \"077ffde3-d76a-472a-b43c-8c3a7c8f9e0e\") " pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh"
Jul 07 15:01:37.858889 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.858853    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:37.971574 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:37.971532    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:38.005685 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:38.005648    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq" event={"ID":"3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3","Type":"ContainerStarted","Data":"63fd561393b51d8f05370c5527a040e8f371f2898162f1fadeee10ae8631d136"}
Jul 07 15:01:38.070603 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:38.070566    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh"
Jul 07 15:01:38.249104 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:38.249062    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-mmcxh"]
Jul 07 15:01:38.252970 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:01:38.252939    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077ffde3_d76a_472a_b43c_8c3a7c8f9e0e.slice/crio-511241135ec70f9eb799c4669bc165d2f791e7b39c01a408a460d30634dee27a WatchSource:0}: Error finding container 511241135ec70f9eb799c4669bc165d2f791e7b39c01a408a460d30634dee27a: Status 404 returned error can't find the container with id 511241135ec70f9eb799c4669bc165d2f791e7b39c01a408a460d30634dee27a
Jul 07 15:01:38.254994 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:38.254977    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:01:39.010587 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:39.010547    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh" event={"ID":"077ffde3-d76a-472a-b43c-8c3a7c8f9e0e","Type":"ContainerStarted","Data":"511241135ec70f9eb799c4669bc165d2f791e7b39c01a408a460d30634dee27a"}
Jul 07 15:01:41.020281 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:41.020244    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq" event={"ID":"3c2e9f71-4282-4ac1-9ba4-3a059d3c66c3","Type":"ContainerStarted","Data":"f8dc2e79d0c7f46b735dcaca1d7fe3ee2b53d4f802aa06e20958242aa537ffb3"}
Jul 07 15:01:41.020709 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:41.020311    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:01:41.021895 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:41.021871    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh" event={"ID":"077ffde3-d76a-472a-b43c-8c3a7c8f9e0e","Type":"ContainerStarted","Data":"0b637018c8fd2cd57bc5793a5a3e62f0a930396da116be8b47b0e69a3a89ae7b"}
Jul 07 15:01:41.022002 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:41.021929    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh"
Jul 07 15:01:41.041392 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:41.041208    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq" podStartSLOduration=2.3124511070000002 podStartE2EDuration="5.041185329s" totalImagesPullingTime="2.728734222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:41.040836311 +0000 UTC m=+304.714442367" watchObservedRunningTime="2026-07-07 15:01:41.041185329 +0000 UTC m=+304.714791385"
Jul 07 15:01:41.058275 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:41.058226    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh" podStartSLOduration=1.359541822 podStartE2EDuration="4.058207589s" totalImagesPullingTime="2.698665767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:41.057842986 +0000 UTC m=+304.731449069" watchObservedRunningTime="2026-07-07 15:01:41.058207589 +0000 UTC m=+304.731813641"
Jul 07 15:01:52.029236 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:01:52.029204    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-d8znq"
Jul 07 15:02:02.028031 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:02.028001    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-mmcxh"
Jul 07 15:02:06.209859 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:06.209817    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:02:08.183911 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:08.183877    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:02:48.834224 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.834110    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:02:48.935567 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.935528    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-c74c8"]
Jul 07 15:02:48.939020 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.938989    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-666b75c8f7-c74c8"
Jul 07 15:02:48.942981 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.942953    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 07 15:02:48.943327 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.943306    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-tk8g4\""
Jul 07 15:02:48.943697 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.943682    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 07 15:02:48.944200 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.944181    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 07 15:02:48.953208 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.953187    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-c74c8"]
Jul 07 15:02:48.961506 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.961483    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-r9nlx"]
Jul 07 15:02:48.964891 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.964869    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-r9nlx"
Jul 07 15:02:48.967763 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.967742    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 07 15:02:48.967868 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.967848    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-jvqqn\""
Jul 07 15:02:48.977512 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:48.977486    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-r9nlx"]
Jul 07 15:02:49.040691 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.040651    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:02:49.043764 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.043743    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/84a1b661-74df-490e-91a7-90ed02cbefe1-data\") pod \"seaweedfs-64568bcd49-r9nlx\" (UID: \"84a1b661-74df-490e-91a7-90ed02cbefe1\") " pod="kserve/seaweedfs-64568bcd49-r9nlx"
Jul 07 15:02:49.043862 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.043785    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x75d5\" (UniqueName: \"kubernetes.io/projected/0f383b3e-79aa-4c0c-b7d2-211a821d6f41-kube-api-access-x75d5\") pod \"kserve-controller-manager-666b75c8f7-c74c8\" (UID: \"0f383b3e-79aa-4c0c-b7d2-211a821d6f41\") " pod="kserve/kserve-controller-manager-666b75c8f7-c74c8"
Jul 07 15:02:49.043862 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.043816    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhq4m\" (UniqueName: \"kubernetes.io/projected/84a1b661-74df-490e-91a7-90ed02cbefe1-kube-api-access-qhq4m\") pod \"seaweedfs-64568bcd49-r9nlx\" (UID: \"84a1b661-74df-490e-91a7-90ed02cbefe1\") " pod="kserve/seaweedfs-64568bcd49-r9nlx"
Jul 07 15:02:49.043955 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.043931    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f383b3e-79aa-4c0c-b7d2-211a821d6f41-cert\") pod \"kserve-controller-manager-666b75c8f7-c74c8\" (UID: \"0f383b3e-79aa-4c0c-b7d2-211a821d6f41\") " pod="kserve/kserve-controller-manager-666b75c8f7-c74c8"
Jul 07 15:02:49.145365 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.145277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x75d5\" (UniqueName: \"kubernetes.io/projected/0f383b3e-79aa-4c0c-b7d2-211a821d6f41-kube-api-access-x75d5\") pod \"kserve-controller-manager-666b75c8f7-c74c8\" (UID: \"0f383b3e-79aa-4c0c-b7d2-211a821d6f41\") " pod="kserve/kserve-controller-manager-666b75c8f7-c74c8"
Jul 07 15:02:49.145365 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.145318    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhq4m\" (UniqueName: \"kubernetes.io/projected/84a1b661-74df-490e-91a7-90ed02cbefe1-kube-api-access-qhq4m\") pod \"seaweedfs-64568bcd49-r9nlx\" (UID: \"84a1b661-74df-490e-91a7-90ed02cbefe1\") " pod="kserve/seaweedfs-64568bcd49-r9nlx"
Jul 07 15:02:49.145560 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.145452    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f383b3e-79aa-4c0c-b7d2-211a821d6f41-cert\") pod \"kserve-controller-manager-666b75c8f7-c74c8\" (UID: \"0f383b3e-79aa-4c0c-b7d2-211a821d6f41\") " pod="kserve/kserve-controller-manager-666b75c8f7-c74c8"
Jul 07 15:02:49.145560 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.145535    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/84a1b661-74df-490e-91a7-90ed02cbefe1-data\") pod \"seaweedfs-64568bcd49-r9nlx\" (UID: \"84a1b661-74df-490e-91a7-90ed02cbefe1\") " pod="kserve/seaweedfs-64568bcd49-r9nlx"
Jul 07 15:02:49.145936 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.145918    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/84a1b661-74df-490e-91a7-90ed02cbefe1-data\") pod \"seaweedfs-64568bcd49-r9nlx\" (UID: \"84a1b661-74df-490e-91a7-90ed02cbefe1\") " pod="kserve/seaweedfs-64568bcd49-r9nlx"
Jul 07 15:02:49.147987 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.147955    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f383b3e-79aa-4c0c-b7d2-211a821d6f41-cert\") pod \"kserve-controller-manager-666b75c8f7-c74c8\" (UID: \"0f383b3e-79aa-4c0c-b7d2-211a821d6f41\") " pod="kserve/kserve-controller-manager-666b75c8f7-c74c8"
Jul 07 15:02:49.158631 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.158607    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhq4m\" (UniqueName: \"kubernetes.io/projected/84a1b661-74df-490e-91a7-90ed02cbefe1-kube-api-access-qhq4m\") pod \"seaweedfs-64568bcd49-r9nlx\" (UID: \"84a1b661-74df-490e-91a7-90ed02cbefe1\") " pod="kserve/seaweedfs-64568bcd49-r9nlx"
Jul 07 15:02:49.168445 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.168420    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x75d5\" (UniqueName: \"kubernetes.io/projected/0f383b3e-79aa-4c0c-b7d2-211a821d6f41-kube-api-access-x75d5\") pod \"kserve-controller-manager-666b75c8f7-c74c8\" (UID: \"0f383b3e-79aa-4c0c-b7d2-211a821d6f41\") " pod="kserve/kserve-controller-manager-666b75c8f7-c74c8"
Jul 07 15:02:49.250660 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.250629    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-666b75c8f7-c74c8"
Jul 07 15:02:49.273576 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.273548    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-r9nlx"
Jul 07 15:02:49.399039 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.398974    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-c74c8"]
Jul 07 15:02:49.401408 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:02:49.401381    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f383b3e_79aa_4c0c_b7d2_211a821d6f41.slice/crio-9e6617243ee92e4f63307a732690f95425e5d0a3fe5ca9c72be235439fb7e9cc WatchSource:0}: Error finding container 9e6617243ee92e4f63307a732690f95425e5d0a3fe5ca9c72be235439fb7e9cc: Status 404 returned error can't find the container with id 9e6617243ee92e4f63307a732690f95425e5d0a3fe5ca9c72be235439fb7e9cc
Jul 07 15:02:49.417862 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.417837    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-r9nlx"]
Jul 07 15:02:49.421011 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:02:49.420984    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a1b661_74df_490e_91a7_90ed02cbefe1.slice/crio-9071397b2ba177eaa0a43d09d180a18bde2dc7cdcc060cb3f82d0ca01dac1f02 WatchSource:0}: Error finding container 9071397b2ba177eaa0a43d09d180a18bde2dc7cdcc060cb3f82d0ca01dac1f02: Status 404 returned error can't find the container with id 9071397b2ba177eaa0a43d09d180a18bde2dc7cdcc060cb3f82d0ca01dac1f02
Jul 07 15:02:49.690332 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:49.690235    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:02:50.024827 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:50.024758    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:02:50.252679 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:50.252639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-r9nlx" event={"ID":"84a1b661-74df-490e-91a7-90ed02cbefe1","Type":"ContainerStarted","Data":"9071397b2ba177eaa0a43d09d180a18bde2dc7cdcc060cb3f82d0ca01dac1f02"}
Jul 07 15:02:50.253867 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:50.253843    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-666b75c8f7-c74c8" event={"ID":"0f383b3e-79aa-4c0c-b7d2-211a821d6f41","Type":"ContainerStarted","Data":"9e6617243ee92e4f63307a732690f95425e5d0a3fe5ca9c72be235439fb7e9cc"}
Jul 07 15:02:53.266611 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:53.266572    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-r9nlx" event={"ID":"84a1b661-74df-490e-91a7-90ed02cbefe1","Type":"ContainerStarted","Data":"fc2c8bf58bac539f4498fd4fe810a4052a93b2e3abd26df93554a7e6615c273f"}
Jul 07 15:02:53.267059 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:53.266651    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-r9nlx"
Jul 07 15:02:53.272793 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:53.269585    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-666b75c8f7-c74c8" event={"ID":"0f383b3e-79aa-4c0c-b7d2-211a821d6f41","Type":"ContainerStarted","Data":"3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710"}
Jul 07 15:02:53.272793 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:53.269884    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-666b75c8f7-c74c8"
Jul 07 15:02:53.291602 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:53.291540    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-r9nlx" podStartSLOduration=1.588642568 podStartE2EDuration="5.291425628s" totalImagesPullingTime="3.70278306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:53.289510688 +0000 UTC m=+376.963116742" watchObservedRunningTime="2026-07-07 15:02:53.291425628 +0000 UTC m=+376.965031661"
Jul 07 15:02:59.275298 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:59.275268    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-r9nlx"
Jul 07 15:02:59.294337 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:02:59.294287    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-666b75c8f7-c74c8" podStartSLOduration=7.659242968 podStartE2EDuration="11.29425157s" totalImagesPullingTime="3.635008602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:53.315475887 +0000 UTC m=+376.989081943" watchObservedRunningTime="2026-07-07 15:02:59.29425157 +0000 UTC m=+382.967857626"
Jul 07 15:03:17.482765 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:17.482726    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:24.278606 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:24.278572    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-666b75c8f7-c74c8"
Jul 07 15:03:33.400244 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.400208    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-c74c8"]
Jul 07 15:03:33.400693 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.400426    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-666b75c8f7-c74c8" podUID="0f383b3e-79aa-4c0c-b7d2-211a821d6f41" containerName="manager" containerID="cri-o://3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710" gracePeriod=10
Jul 07 15:03:33.425661 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.425634    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-llgnq"]
Jul 07 15:03:33.428736 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.428715    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-666b75c8f7-llgnq"
Jul 07 15:03:33.440785 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.440759    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-llgnq"]
Jul 07 15:03:33.592441 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.592410    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbz48\" (UniqueName: \"kubernetes.io/projected/f787152a-b959-49dd-b3c8-9627ab5a4b67-kube-api-access-fbz48\") pod \"kserve-controller-manager-666b75c8f7-llgnq\" (UID: \"f787152a-b959-49dd-b3c8-9627ab5a4b67\") " pod="kserve/kserve-controller-manager-666b75c8f7-llgnq"
Jul 07 15:03:33.592601 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.592455    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f787152a-b959-49dd-b3c8-9627ab5a4b67-cert\") pod \"kserve-controller-manager-666b75c8f7-llgnq\" (UID: \"f787152a-b959-49dd-b3c8-9627ab5a4b67\") " pod="kserve/kserve-controller-manager-666b75c8f7-llgnq"
Jul 07 15:03:33.635952 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.635929    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-666b75c8f7-c74c8"
Jul 07 15:03:33.693683 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.693591    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbz48\" (UniqueName: \"kubernetes.io/projected/f787152a-b959-49dd-b3c8-9627ab5a4b67-kube-api-access-fbz48\") pod \"kserve-controller-manager-666b75c8f7-llgnq\" (UID: \"f787152a-b959-49dd-b3c8-9627ab5a4b67\") " pod="kserve/kserve-controller-manager-666b75c8f7-llgnq"
Jul 07 15:03:33.693683 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.693645    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f787152a-b959-49dd-b3c8-9627ab5a4b67-cert\") pod \"kserve-controller-manager-666b75c8f7-llgnq\" (UID: \"f787152a-b959-49dd-b3c8-9627ab5a4b67\") " pod="kserve/kserve-controller-manager-666b75c8f7-llgnq"
Jul 07 15:03:33.695950 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.695927    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f787152a-b959-49dd-b3c8-9627ab5a4b67-cert\") pod \"kserve-controller-manager-666b75c8f7-llgnq\" (UID: \"f787152a-b959-49dd-b3c8-9627ab5a4b67\") " pod="kserve/kserve-controller-manager-666b75c8f7-llgnq"
Jul 07 15:03:33.708732 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.708703    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbz48\" (UniqueName: \"kubernetes.io/projected/f787152a-b959-49dd-b3c8-9627ab5a4b67-kube-api-access-fbz48\") pod \"kserve-controller-manager-666b75c8f7-llgnq\" (UID: \"f787152a-b959-49dd-b3c8-9627ab5a4b67\") " pod="kserve/kserve-controller-manager-666b75c8f7-llgnq"
Jul 07 15:03:33.780474 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.780428    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-666b75c8f7-llgnq"
Jul 07 15:03:33.794876 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.794848    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f383b3e-79aa-4c0c-b7d2-211a821d6f41-cert\") pod \"0f383b3e-79aa-4c0c-b7d2-211a821d6f41\" (UID: \"0f383b3e-79aa-4c0c-b7d2-211a821d6f41\") "
Jul 07 15:03:33.794997 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.794899    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x75d5\" (UniqueName: \"kubernetes.io/projected/0f383b3e-79aa-4c0c-b7d2-211a821d6f41-kube-api-access-x75d5\") pod \"0f383b3e-79aa-4c0c-b7d2-211a821d6f41\" (UID: \"0f383b3e-79aa-4c0c-b7d2-211a821d6f41\") "
Jul 07 15:03:33.797013 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.796985    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f383b3e-79aa-4c0c-b7d2-211a821d6f41-kube-api-access-x75d5" (OuterVolumeSpecName: "kube-api-access-x75d5") pod "0f383b3e-79aa-4c0c-b7d2-211a821d6f41" (UID: "0f383b3e-79aa-4c0c-b7d2-211a821d6f41"). InnerVolumeSpecName "kube-api-access-x75d5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:03:33.797089 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.797002    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f383b3e-79aa-4c0c-b7d2-211a821d6f41-cert" (OuterVolumeSpecName: "cert") pod "0f383b3e-79aa-4c0c-b7d2-211a821d6f41" (UID: "0f383b3e-79aa-4c0c-b7d2-211a821d6f41"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:03:33.895719 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.895686    2582 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f383b3e-79aa-4c0c-b7d2-211a821d6f41-cert\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:03:33.895719 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.895716    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x75d5\" (UniqueName: \"kubernetes.io/projected/0f383b3e-79aa-4c0c-b7d2-211a821d6f41-kube-api-access-x75d5\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:03:33.909795 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:33.909757    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-llgnq"]
Jul 07 15:03:33.913668 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:03:33.913635    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf787152a_b959_49dd_b3c8_9627ab5a4b67.slice/crio-27f88f362489874f634dfc35cb2cdc0c3789fd58ed4d6e43e165a4f4242d0659 WatchSource:0}: Error finding container 27f88f362489874f634dfc35cb2cdc0c3789fd58ed4d6e43e165a4f4242d0659: Status 404 returned error can't find the container with id 27f88f362489874f634dfc35cb2cdc0c3789fd58ed4d6e43e165a4f4242d0659
Jul 07 15:03:34.410888 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:34.410855    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-666b75c8f7-llgnq" event={"ID":"f787152a-b959-49dd-b3c8-9627ab5a4b67","Type":"ContainerStarted","Data":"27f88f362489874f634dfc35cb2cdc0c3789fd58ed4d6e43e165a4f4242d0659"}
Jul 07 15:03:34.412074 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:34.412048    2582 generic.go:356] "Generic (PLEG): container finished" podID="0f383b3e-79aa-4c0c-b7d2-211a821d6f41" containerID="3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710" exitCode=0
Jul 07 15:03:34.412194 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:34.412101    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-666b75c8f7-c74c8"
Jul 07 15:03:34.412194 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:34.412144    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-666b75c8f7-c74c8" event={"ID":"0f383b3e-79aa-4c0c-b7d2-211a821d6f41","Type":"ContainerDied","Data":"3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710"}
Jul 07 15:03:34.412318 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:34.412203    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-666b75c8f7-c74c8" event={"ID":"0f383b3e-79aa-4c0c-b7d2-211a821d6f41","Type":"ContainerDied","Data":"9e6617243ee92e4f63307a732690f95425e5d0a3fe5ca9c72be235439fb7e9cc"}
Jul 07 15:03:34.412318 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:34.412236    2582 scope.go:117] "RemoveContainer" containerID="3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710"
Jul 07 15:03:34.421453 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:34.421437    2582 scope.go:117] "RemoveContainer" containerID="3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710"
Jul 07 15:03:34.421693 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:03:34.421675    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710\": container with ID starting with 3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710 not found: ID does not exist" containerID="3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710"
Jul 07 15:03:34.421740 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:34.421701    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710"} err="failed to get container status \"3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710\": rpc error: code = NotFound desc = could not find container \"3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710\": container with ID starting with 3dbd7cc215faa5410dc04a8c3646f0373242bb6fe8f906caf8effffdde47c710 not found: ID does not exist"
Jul 07 15:03:34.446374 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:34.446351    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-c74c8"]
Jul 07 15:03:34.452229 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:34.452210    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-666b75c8f7-c74c8"]
Jul 07 15:03:34.791580 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:34.791549    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f383b3e-79aa-4c0c-b7d2-211a821d6f41" path="/var/lib/kubelet/pods/0f383b3e-79aa-4c0c-b7d2-211a821d6f41/volumes"
Jul 07 15:03:35.416770 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:35.416737    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-666b75c8f7-llgnq" event={"ID":"f787152a-b959-49dd-b3c8-9627ab5a4b67","Type":"ContainerStarted","Data":"fc8fa4ca5ee7b311e8fb151bf861670b5ab6da05197ce3bfe103b8757c33fd32"}
Jul 07 15:03:35.417236 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:35.416904    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-666b75c8f7-llgnq"
Jul 07 15:03:35.438422 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:35.438364    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-666b75c8f7-llgnq" podStartSLOduration=1.461071765 podStartE2EDuration="2.438345925s" totalImagesPullingTime="977.27416ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:03:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:03:35.437013605 +0000 UTC m=+419.110619659" watchObservedRunningTime="2026-07-07 15:03:35.438345925 +0000 UTC m=+419.111951983"
Jul 07 15:03:58.708362 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:58.708326    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:58.771397 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:03:58.771321    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:04:00.803554 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:00.803523    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:04:00.861986 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:00.861963    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:04:06.426282 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:06.426251    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-666b75c8f7-llgnq"
Jul 07 15:04:07.372884 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.372851    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:04:07.431022 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.430990    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-gp5j5"]
Jul 07 15:04:07.431499 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.431276    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f383b3e-79aa-4c0c-b7d2-211a821d6f41" containerName="manager"
Jul 07 15:04:07.431499 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.431288    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f383b3e-79aa-4c0c-b7d2-211a821d6f41" containerName="manager"
Jul 07 15:04:07.431499 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.431349    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f383b3e-79aa-4c0c-b7d2-211a821d6f41" containerName="manager"
Jul 07 15:04:07.434085 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.434069    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-gp5j5"
Jul 07 15:04:07.437333 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.437309    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jul 07 15:04:07.437454 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.437370    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-qprfg\""
Jul 07 15:04:07.445513 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.445484    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-gp5j5"]
Jul 07 15:04:07.450087 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.450060    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:04:07.452229 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.452200    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxftf\" (UniqueName: \"kubernetes.io/projected/ba5555f8-b28e-44e2-877f-96c18c8dcbb0-kube-api-access-zxftf\") pod \"odh-model-controller-84b54b7f97-gp5j5\" (UID: \"ba5555f8-b28e-44e2-877f-96c18c8dcbb0\") " pod="kserve/odh-model-controller-84b54b7f97-gp5j5"
Jul 07 15:04:07.452343 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.452246    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba5555f8-b28e-44e2-877f-96c18c8dcbb0-cert\") pod \"odh-model-controller-84b54b7f97-gp5j5\" (UID: \"ba5555f8-b28e-44e2-877f-96c18c8dcbb0\") " pod="kserve/odh-model-controller-84b54b7f97-gp5j5"
Jul 07 15:04:07.552891 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.552856    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxftf\" (UniqueName: \"kubernetes.io/projected/ba5555f8-b28e-44e2-877f-96c18c8dcbb0-kube-api-access-zxftf\") pod \"odh-model-controller-84b54b7f97-gp5j5\" (UID: \"ba5555f8-b28e-44e2-877f-96c18c8dcbb0\") " pod="kserve/odh-model-controller-84b54b7f97-gp5j5"
Jul 07 15:04:07.552891 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.552895    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba5555f8-b28e-44e2-877f-96c18c8dcbb0-cert\") pod \"odh-model-controller-84b54b7f97-gp5j5\" (UID: \"ba5555f8-b28e-44e2-877f-96c18c8dcbb0\") " pod="kserve/odh-model-controller-84b54b7f97-gp5j5"
Jul 07 15:04:07.553172 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:04:07.552985    2582 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 07 15:04:07.553172 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:04:07.553034    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba5555f8-b28e-44e2-877f-96c18c8dcbb0-cert podName:ba5555f8-b28e-44e2-877f-96c18c8dcbb0 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:08.053020157 +0000 UTC m=+451.726626190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ba5555f8-b28e-44e2-877f-96c18c8dcbb0-cert") pod "odh-model-controller-84b54b7f97-gp5j5" (UID: "ba5555f8-b28e-44e2-877f-96c18c8dcbb0") : secret "odh-model-controller-webhook-cert" not found
Jul 07 15:04:07.562294 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.562268    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxftf\" (UniqueName: \"kubernetes.io/projected/ba5555f8-b28e-44e2-877f-96c18c8dcbb0-kube-api-access-zxftf\") pod \"odh-model-controller-84b54b7f97-gp5j5\" (UID: \"ba5555f8-b28e-44e2-877f-96c18c8dcbb0\") " pod="kserve/odh-model-controller-84b54b7f97-gp5j5"
Jul 07 15:04:07.597404 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.597368    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:04:07.705411 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:07.705328    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:04:08.057290 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:08.057254    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba5555f8-b28e-44e2-877f-96c18c8dcbb0-cert\") pod \"odh-model-controller-84b54b7f97-gp5j5\" (UID: \"ba5555f8-b28e-44e2-877f-96c18c8dcbb0\") " pod="kserve/odh-model-controller-84b54b7f97-gp5j5"
Jul 07 15:04:08.059489 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:08.059462    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba5555f8-b28e-44e2-877f-96c18c8dcbb0-cert\") pod \"odh-model-controller-84b54b7f97-gp5j5\" (UID: \"ba5555f8-b28e-44e2-877f-96c18c8dcbb0\") " pod="kserve/odh-model-controller-84b54b7f97-gp5j5"
Jul 07 15:04:08.141353 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:08.141325    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:04:08.198183 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:08.198124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:04:08.345023 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:08.344940    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-gp5j5"
Jul 07 15:04:08.476274 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:08.476251    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-gp5j5"]
Jul 07 15:04:08.478463 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:04:08.478435    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5555f8_b28e_44e2_877f_96c18c8dcbb0.slice/crio-c0f741625842cb30c8cf9a6422a98916ebfdd153b4882bb5f4e8f5a0d2d84e3c WatchSource:0}: Error finding container c0f741625842cb30c8cf9a6422a98916ebfdd153b4882bb5f4e8f5a0d2d84e3c: Status 404 returned error can't find the container with id c0f741625842cb30c8cf9a6422a98916ebfdd153b4882bb5f4e8f5a0d2d84e3c
Jul 07 15:04:08.534046 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:08.534012    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-gp5j5" event={"ID":"ba5555f8-b28e-44e2-877f-96c18c8dcbb0","Type":"ContainerStarted","Data":"c0f741625842cb30c8cf9a6422a98916ebfdd153b4882bb5f4e8f5a0d2d84e3c"}
Jul 07 15:04:11.548731 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:11.548690    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-gp5j5" event={"ID":"ba5555f8-b28e-44e2-877f-96c18c8dcbb0","Type":"ContainerStarted","Data":"8f5f8ff05fd67aaa936594c4a4265950e4a5d9ef6586038da7db8a51b96938f1"}
Jul 07 15:04:11.549245 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:11.548800    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-gp5j5"
Jul 07 15:04:11.569114 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:11.569060    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-gp5j5" podStartSLOduration=1.899607853 podStartE2EDuration="4.569046643s" totalImagesPullingTime="2.66943879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:04:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:04:11.568246398 +0000 UTC m=+455.241852450" watchObservedRunningTime="2026-07-07 15:04:11.569046643 +0000 UTC m=+455.242652723"
Jul 07 15:04:22.556110 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:04:22.556075    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-gp5j5"
Jul 07 15:05:43.995224 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:43.995122    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:51.664011 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.663974    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"]
Jul 07 15:05:51.667118 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.667100    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"
Jul 07 15:05:51.671696 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.671674    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 07 15:05:51.671959 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.671944    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom\""
Jul 07 15:05:51.687614 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.687587    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"]
Jul 07 15:05:51.724282 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.724243    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c945218a-1cd3-47d1-aceb-4bf07519db2f-data\") pod \"seaweedfs-tls-custom-8b44d767d-c5xc5\" (UID: \"c945218a-1cd3-47d1-aceb-4bf07519db2f\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"
Jul 07 15:05:51.724282 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.724282    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf97v\" (UniqueName: \"kubernetes.io/projected/c945218a-1cd3-47d1-aceb-4bf07519db2f-kube-api-access-vf97v\") pod \"seaweedfs-tls-custom-8b44d767d-c5xc5\" (UID: \"c945218a-1cd3-47d1-aceb-4bf07519db2f\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"
Jul 07 15:05:51.724503 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.724316    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/c945218a-1cd3-47d1-aceb-4bf07519db2f-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-c5xc5\" (UID: \"c945218a-1cd3-47d1-aceb-4bf07519db2f\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"
Jul 07 15:05:51.825119 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.825075    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c945218a-1cd3-47d1-aceb-4bf07519db2f-data\") pod \"seaweedfs-tls-custom-8b44d767d-c5xc5\" (UID: \"c945218a-1cd3-47d1-aceb-4bf07519db2f\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"
Jul 07 15:05:51.825313 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.825121    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf97v\" (UniqueName: \"kubernetes.io/projected/c945218a-1cd3-47d1-aceb-4bf07519db2f-kube-api-access-vf97v\") pod \"seaweedfs-tls-custom-8b44d767d-c5xc5\" (UID: \"c945218a-1cd3-47d1-aceb-4bf07519db2f\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"
Jul 07 15:05:51.825313 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.825203    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/c945218a-1cd3-47d1-aceb-4bf07519db2f-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-c5xc5\" (UID: \"c945218a-1cd3-47d1-aceb-4bf07519db2f\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"
Jul 07 15:05:51.825563 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.825540    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c945218a-1cd3-47d1-aceb-4bf07519db2f-data\") pod \"seaweedfs-tls-custom-8b44d767d-c5xc5\" (UID: \"c945218a-1cd3-47d1-aceb-4bf07519db2f\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"
Jul 07 15:05:51.827706 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.827688    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/c945218a-1cd3-47d1-aceb-4bf07519db2f-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-c5xc5\" (UID: \"c945218a-1cd3-47d1-aceb-4bf07519db2f\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"
Jul 07 15:05:51.834582 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.834556    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf97v\" (UniqueName: \"kubernetes.io/projected/c945218a-1cd3-47d1-aceb-4bf07519db2f-kube-api-access-vf97v\") pod \"seaweedfs-tls-custom-8b44d767d-c5xc5\" (UID: \"c945218a-1cd3-47d1-aceb-4bf07519db2f\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"
Jul 07 15:05:51.975878 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:51.975849    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"
Jul 07 15:05:52.122798 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:52.122775    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-c5xc5"]
Jul 07 15:05:52.124528 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:05:52.124499    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc945218a_1cd3_47d1_aceb_4bf07519db2f.slice/crio-3d17b6ac4cdbb8cfb1ccc167d76153b6889366450ac83a2d2e2b4681b311d688 WatchSource:0}: Error finding container 3d17b6ac4cdbb8cfb1ccc167d76153b6889366450ac83a2d2e2b4681b311d688: Status 404 returned error can't find the container with id 3d17b6ac4cdbb8cfb1ccc167d76153b6889366450ac83a2d2e2b4681b311d688
Jul 07 15:05:52.916024 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:52.915991    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5" event={"ID":"c945218a-1cd3-47d1-aceb-4bf07519db2f","Type":"ContainerStarted","Data":"fbe785d5e1c9638dabb80db24371334d3088a87763537ab5d29ed3bd4e724ca6"}
Jul 07 15:05:52.916418 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:52.916041    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5" event={"ID":"c945218a-1cd3-47d1-aceb-4bf07519db2f","Type":"ContainerStarted","Data":"3d17b6ac4cdbb8cfb1ccc167d76153b6889366450ac83a2d2e2b4681b311d688"}
Jul 07 15:05:52.936252 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:05:52.936176    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-8b44d767d-c5xc5" podStartSLOduration=1.6830912279999999 podStartE2EDuration="1.936155902s" totalImagesPullingTime="253.064674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:52.935224643 +0000 UTC m=+556.608830699" watchObservedRunningTime="2026-07-07 15:05:52.936155902 +0000 UTC m=+556.609761957"
Jul 07 15:06:04.449003 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:04.448965    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:04.818854 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:04.818824    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:18.706628 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:18.706588    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:19.206595 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:19.206562    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:19.310026 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:19.309993    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:31.364832 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.364789    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"]
Jul 07 15:06:31.367755 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.367737    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.373319 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.373279    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 07 15:06:31.373319 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.373306    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 07 15:06:31.373535 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.373374    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 07 15:06:31.373637 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.373617    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-bnv6j\""
Jul 07 15:06:31.374171 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.374151    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-kube-rbac-proxy-sar-config\""
Jul 07 15:06:31.374875 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.374854    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-predictor-serving-cert\""
Jul 07 15:06:31.380496 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.380476    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 07 15:06:31.402520 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.402494    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"]
Jul 07 15:06:31.411922 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.411898    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/713fd33d-600f-4b2d-bcc8-adb3855efe06-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.412025 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.411932    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqzgl\" (UniqueName: \"kubernetes.io/projected/713fd33d-600f-4b2d-bcc8-adb3855efe06-kube-api-access-hqzgl\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.412025 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.411951    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/713fd33d-600f-4b2d-bcc8-adb3855efe06-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.412025 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.411978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/713fd33d-600f-4b2d-bcc8-adb3855efe06-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.412152 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.412054    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/713fd33d-600f-4b2d-bcc8-adb3855efe06-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.513429 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.513394    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/713fd33d-600f-4b2d-bcc8-adb3855efe06-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.513591 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.513438    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqzgl\" (UniqueName: \"kubernetes.io/projected/713fd33d-600f-4b2d-bcc8-adb3855efe06-kube-api-access-hqzgl\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.513591 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.513460    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/713fd33d-600f-4b2d-bcc8-adb3855efe06-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.513591 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.513499    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/713fd33d-600f-4b2d-bcc8-adb3855efe06-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.513591 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.513549    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/713fd33d-600f-4b2d-bcc8-adb3855efe06-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.513905 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.513881    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/713fd33d-600f-4b2d-bcc8-adb3855efe06-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.514215 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.514196    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/713fd33d-600f-4b2d-bcc8-adb3855efe06-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.514260 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.514204    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/713fd33d-600f-4b2d-bcc8-adb3855efe06-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.515885 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.515869    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/713fd33d-600f-4b2d-bcc8-adb3855efe06-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.534285 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.534261    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqzgl\" (UniqueName: \"kubernetes.io/projected/713fd33d-600f-4b2d-bcc8-adb3855efe06-kube-api-access-hqzgl\") pod \"isvc-sklearn-batcher-predictor-956c77b89-btfqw\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.679190 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.679067    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:06:31.690061 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.690036    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:31.780147 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.780083    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:31.817326 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:31.817151    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"]
Jul 07 15:06:31.819960 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:06:31.819933    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod713fd33d_600f_4b2d_bcc8_adb3855efe06.slice/crio-b408fef18ec7e3df6a24c6882429a27d1447905b7de858500b5621ecd6f619bd WatchSource:0}: Error finding container b408fef18ec7e3df6a24c6882429a27d1447905b7de858500b5621ecd6f619bd: Status 404 returned error can't find the container with id b408fef18ec7e3df6a24c6882429a27d1447905b7de858500b5621ecd6f619bd
Jul 07 15:06:32.057564 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:32.057526    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" event={"ID":"713fd33d-600f-4b2d-bcc8-adb3855efe06","Type":"ContainerStarted","Data":"b408fef18ec7e3df6a24c6882429a27d1447905b7de858500b5621ecd6f619bd"}
Jul 07 15:06:36.075203 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:36.075163    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" event={"ID":"713fd33d-600f-4b2d-bcc8-adb3855efe06","Type":"ContainerStarted","Data":"27bd4c7ad98005c8a114643b0db92f8ac7631b9a983046ccbe37a0fc017c7451"}
Jul 07 15:06:40.091632 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:40.091597    2582 generic.go:356] "Generic (PLEG): container finished" podID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerID="27bd4c7ad98005c8a114643b0db92f8ac7631b9a983046ccbe37a0fc017c7451" exitCode=0
Jul 07 15:06:40.092177 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:40.091678    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" event={"ID":"713fd33d-600f-4b2d-bcc8-adb3855efe06","Type":"ContainerDied","Data":"27bd4c7ad98005c8a114643b0db92f8ac7631b9a983046ccbe37a0fc017c7451"}
Jul 07 15:06:40.092916 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:40.092901    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:06:54.161589 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:54.161551    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" event={"ID":"713fd33d-600f-4b2d-bcc8-adb3855efe06","Type":"ContainerStarted","Data":"a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07"}
Jul 07 15:06:56.173741 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:06:56.173652    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" event={"ID":"713fd33d-600f-4b2d-bcc8-adb3855efe06","Type":"ContainerStarted","Data":"f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1"}
Jul 07 15:07:00.191874 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:00.191832    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" event={"ID":"713fd33d-600f-4b2d-bcc8-adb3855efe06","Type":"ContainerStarted","Data":"1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf"}
Jul 07 15:07:00.192357 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:00.192069    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:07:00.197679 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:00.197653    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:07:00.223996 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:00.223945    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podStartSLOduration=2.464254008 podStartE2EDuration="29.223928606s" totalImagesPullingTime="22.948735147s" totalInitContainerRuntime="3.810939451s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:06:31 +0000 UTC" imagePullSessionsCount=4 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:07:00.222786932 +0000 UTC m=+623.896392999" watchObservedRunningTime="2026-07-07 15:07:00.223928606 +0000 UTC m=+623.897534661"
Jul 07 15:07:01.195705 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:01.195665    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:07:01.196099 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:01.195749    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:07:01.196099 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:01.195988    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 07 15:07:01.196593 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:01.196564    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:02.198863 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:02.198818    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 07 15:07:02.199266 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:02.199178    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:07.563302 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:07.563268    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:07.638463 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:07.638425    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:12.199659 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:12.199606    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 07 15:07:12.200162 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:12.200022    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:14.980276 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:14.980243    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:22.199232 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:22.199172    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 07 15:07:22.199720 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:22.199635    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:32.199080 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:32.199037    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 07 15:07:32.199596 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:32.199464    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:42.199273 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:42.199220    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 07 15:07:42.199735 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:42.199577    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:52.199693 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:52.199650    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 07 15:07:52.200187 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:52.200032    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:57.500455 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:57.500418    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:57.602478 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:07:57.602431    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:08:02.199753 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:02.199703    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 07 15:08:02.200219 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:02.200178    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:08:12.200100 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:12.200066    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:08:12.200502 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:12.200283    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:08:26.195243 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.195209    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:08:26.280187 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.280152    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"]
Jul 07 15:08:26.280449 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.280414    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container" containerID="cri-o://a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07" gracePeriod=30
Jul 07 15:08:26.280601 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.280426    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent" containerID="cri-o://1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf" gracePeriod=30
Jul 07 15:08:26.280601 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.280509    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kube-rbac-proxy" containerID="cri-o://f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1" gracePeriod=30
Jul 07 15:08:26.285316 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.285279    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:08:26.292226 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.292194    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"]
Jul 07 15:08:26.295918 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.295895    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.300177 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.300159    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-predictor-serving-cert\""
Jul 07 15:08:26.300892 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.300876    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-kube-rbac-proxy-sar-config\""
Jul 07 15:08:26.310979 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.310952    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"]
Jul 07 15:08:26.343839 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.343794    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4848a366-b814-4b63-920a-0f3edd01a0e5-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.344058 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.343918    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4848a366-b814-4b63-920a-0f3edd01a0e5-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.344058 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.343957    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4848a366-b814-4b63-920a-0f3edd01a0e5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.344058 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.344014    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsds9\" (UniqueName: \"kubernetes.io/projected/4848a366-b814-4b63-920a-0f3edd01a0e5-kube-api-access-hsds9\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.344244 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.344163    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4848a366-b814-4b63-920a-0f3edd01a0e5-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.348379 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.348355    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:08:26.445185 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.445150    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4848a366-b814-4b63-920a-0f3edd01a0e5-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.445368 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.445202    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4848a366-b814-4b63-920a-0f3edd01a0e5-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.445368 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.445242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4848a366-b814-4b63-920a-0f3edd01a0e5-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.445368 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.445265    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4848a366-b814-4b63-920a-0f3edd01a0e5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.445368 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.445301    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsds9\" (UniqueName: \"kubernetes.io/projected/4848a366-b814-4b63-920a-0f3edd01a0e5-kube-api-access-hsds9\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.445631 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.445604    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4848a366-b814-4b63-920a-0f3edd01a0e5-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.445921 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.445897    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4848a366-b814-4b63-920a-0f3edd01a0e5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.446039 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.445924    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4848a366-b814-4b63-920a-0f3edd01a0e5-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.447903 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.447873    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4848a366-b814-4b63-920a-0f3edd01a0e5-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.455031 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.455006    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsds9\" (UniqueName: \"kubernetes.io/projected/4848a366-b814-4b63-920a-0f3edd01a0e5-kube-api-access-hsds9\") pod \"isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.518553 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.518517    2582 generic.go:356] "Generic (PLEG): container finished" podID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerID="f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1" exitCode=2
Jul 07 15:08:26.518553 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.518561    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" event={"ID":"713fd33d-600f-4b2d-bcc8-adb3855efe06","Type":"ContainerDied","Data":"f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1"}
Jul 07 15:08:26.606558 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.606512    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:26.752949 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:26.752844    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"]
Jul 07 15:08:26.755884 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:08:26.755855    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4848a366_b814_4b63_920a_0f3edd01a0e5.slice/crio-65aabfb4ffa524b48cb020d16aecfae1f789301b16ced88808ee438498da0061 WatchSource:0}: Error finding container 65aabfb4ffa524b48cb020d16aecfae1f789301b16ced88808ee438498da0061: Status 404 returned error can't find the container with id 65aabfb4ffa524b48cb020d16aecfae1f789301b16ced88808ee438498da0061
Jul 07 15:08:27.523696 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:27.523661    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" event={"ID":"4848a366-b814-4b63-920a-0f3edd01a0e5","Type":"ContainerStarted","Data":"a21ed028dc6bc2684ccd979c6eab4094268fd006a5cc8698f21a7f31a2919d10"}
Jul 07 15:08:27.524156 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:27.523729    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" event={"ID":"4848a366-b814-4b63-920a-0f3edd01a0e5","Type":"ContainerStarted","Data":"65aabfb4ffa524b48cb020d16aecfae1f789301b16ced88808ee438498da0061"}
Jul 07 15:08:30.192773 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:30.192677    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Jul 07 15:08:31.540753 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:31.540718    2582 generic.go:356] "Generic (PLEG): container finished" podID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerID="a21ed028dc6bc2684ccd979c6eab4094268fd006a5cc8698f21a7f31a2919d10" exitCode=0
Jul 07 15:08:31.541215 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:31.540792    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" event={"ID":"4848a366-b814-4b63-920a-0f3edd01a0e5","Type":"ContainerDied","Data":"a21ed028dc6bc2684ccd979c6eab4094268fd006a5cc8698f21a7f31a2919d10"}
Jul 07 15:08:31.543342 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:31.543315    2582 generic.go:356] "Generic (PLEG): container finished" podID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerID="a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07" exitCode=0
Jul 07 15:08:31.543463 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:31.543370    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" event={"ID":"713fd33d-600f-4b2d-bcc8-adb3855efe06","Type":"ContainerDied","Data":"a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07"}
Jul 07 15:08:32.198942 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:32.198889    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 07 15:08:32.199286 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:32.199258    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:08:32.550551 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:32.550521    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" event={"ID":"4848a366-b814-4b63-920a-0f3edd01a0e5","Type":"ContainerStarted","Data":"58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09"}
Jul 07 15:08:32.550939 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:32.550587    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" event={"ID":"4848a366-b814-4b63-920a-0f3edd01a0e5","Type":"ContainerStarted","Data":"c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9"}
Jul 07 15:08:32.550939 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:32.550611    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" event={"ID":"4848a366-b814-4b63-920a-0f3edd01a0e5","Type":"ContainerStarted","Data":"5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787"}
Jul 07 15:08:32.550939 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:32.550900    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:32.551051 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:32.551035    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:32.552334 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:32.552307    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:08:32.575371 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:32.575316    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podStartSLOduration=2.839458116 podStartE2EDuration="6.575301267s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.735843151s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:08:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:08:32.573274504 +0000 UTC m=+716.246880560" watchObservedRunningTime="2026-07-07 15:08:32.575301267 +0000 UTC m=+716.248907322"
Jul 07 15:08:33.559059 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:33.557444    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:33.559445 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:33.559122    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:08:33.559853 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:33.559812    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:08:34.560617 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:34.560575    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:08:34.561048 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:34.561023    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:08:35.193066 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:35.193012    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Jul 07 15:08:39.566256 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:39.566223    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:08:39.566937 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:39.566903    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:08:39.567210 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:39.567180    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:08:40.192786 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:40.192735    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Jul 07 15:08:40.192976 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:40.192915    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:08:42.199187 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:42.199111    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 07 15:08:42.199621 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:42.199448    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:08:45.192921 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:45.192874    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Jul 07 15:08:49.567396 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:49.567348    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:08:49.567914 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:49.567888    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:08:50.192787 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:50.192745    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Jul 07 15:08:52.199764 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:52.199709    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 07 15:08:52.200185 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:52.199858    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:08:52.200185 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:52.200024    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:08:52.200265 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:52.200184    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:08:55.192510 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:55.192465    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Jul 07 15:08:56.432377 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.432352    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:08:56.600059 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.599971    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqzgl\" (UniqueName: \"kubernetes.io/projected/713fd33d-600f-4b2d-bcc8-adb3855efe06-kube-api-access-hqzgl\") pod \"713fd33d-600f-4b2d-bcc8-adb3855efe06\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") "
Jul 07 15:08:56.600059 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.600029    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/713fd33d-600f-4b2d-bcc8-adb3855efe06-cabundle-cert\") pod \"713fd33d-600f-4b2d-bcc8-adb3855efe06\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") "
Jul 07 15:08:56.600059 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.600051    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/713fd33d-600f-4b2d-bcc8-adb3855efe06-kube-rbac-proxy-sar-config\") pod \"713fd33d-600f-4b2d-bcc8-adb3855efe06\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") "
Jul 07 15:08:56.600363 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.600099    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/713fd33d-600f-4b2d-bcc8-adb3855efe06-kserve-provision-location\") pod \"713fd33d-600f-4b2d-bcc8-adb3855efe06\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") "
Jul 07 15:08:56.600363 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.600159    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/713fd33d-600f-4b2d-bcc8-adb3855efe06-proxy-tls\") pod \"713fd33d-600f-4b2d-bcc8-adb3855efe06\" (UID: \"713fd33d-600f-4b2d-bcc8-adb3855efe06\") "
Jul 07 15:08:56.600526 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.600498    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/713fd33d-600f-4b2d-bcc8-adb3855efe06-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "713fd33d-600f-4b2d-bcc8-adb3855efe06" (UID: "713fd33d-600f-4b2d-bcc8-adb3855efe06"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:08:56.600526 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.600514    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/713fd33d-600f-4b2d-bcc8-adb3855efe06-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "713fd33d-600f-4b2d-bcc8-adb3855efe06" (UID: "713fd33d-600f-4b2d-bcc8-adb3855efe06"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:08:56.600620 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.600511    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/713fd33d-600f-4b2d-bcc8-adb3855efe06-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "713fd33d-600f-4b2d-bcc8-adb3855efe06" (UID: "713fd33d-600f-4b2d-bcc8-adb3855efe06"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:08:56.602265 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.602237    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/713fd33d-600f-4b2d-bcc8-adb3855efe06-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "713fd33d-600f-4b2d-bcc8-adb3855efe06" (UID: "713fd33d-600f-4b2d-bcc8-adb3855efe06"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:08:56.602351 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.602326    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/713fd33d-600f-4b2d-bcc8-adb3855efe06-kube-api-access-hqzgl" (OuterVolumeSpecName: "kube-api-access-hqzgl") pod "713fd33d-600f-4b2d-bcc8-adb3855efe06" (UID: "713fd33d-600f-4b2d-bcc8-adb3855efe06"). InnerVolumeSpecName "kube-api-access-hqzgl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:08:56.648276 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.648244    2582 generic.go:356] "Generic (PLEG): container finished" podID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerID="1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf" exitCode=0
Jul 07 15:08:56.648476 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.648315    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"
Jul 07 15:08:56.648476 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.648317    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" event={"ID":"713fd33d-600f-4b2d-bcc8-adb3855efe06","Type":"ContainerDied","Data":"1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf"}
Jul 07 15:08:56.648476 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.648449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw" event={"ID":"713fd33d-600f-4b2d-bcc8-adb3855efe06","Type":"ContainerDied","Data":"b408fef18ec7e3df6a24c6882429a27d1447905b7de858500b5621ecd6f619bd"}
Jul 07 15:08:56.648621 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.648480    2582 scope.go:117] "RemoveContainer" containerID="1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf"
Jul 07 15:08:56.656683 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.656589    2582 scope.go:117] "RemoveContainer" containerID="f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1"
Jul 07 15:08:56.664089 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.664071    2582 scope.go:117] "RemoveContainer" containerID="a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07"
Jul 07 15:08:56.670991 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.670970    2582 scope.go:117] "RemoveContainer" containerID="27bd4c7ad98005c8a114643b0db92f8ac7631b9a983046ccbe37a0fc017c7451"
Jul 07 15:08:56.674396 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.674376    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"]
Jul 07 15:08:56.678983 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.678960    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-956c77b89-btfqw"]
Jul 07 15:08:56.679404 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.679390    2582 scope.go:117] "RemoveContainer" containerID="1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf"
Jul 07 15:08:56.679685 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:08:56.679668    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf\": container with ID starting with 1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf not found: ID does not exist" containerID="1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf"
Jul 07 15:08:56.679738 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.679695    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf"} err="failed to get container status \"1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf\": rpc error: code = NotFound desc = could not find container \"1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf\": container with ID starting with 1cfd24758a3822fefafbc87cda960d8793daf92d6b29e1b68a836658502f02bf not found: ID does not exist"
Jul 07 15:08:56.679738 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.679714    2582 scope.go:117] "RemoveContainer" containerID="f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1"
Jul 07 15:08:56.679937 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:08:56.679921    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1\": container with ID starting with f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1 not found: ID does not exist" containerID="f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1"
Jul 07 15:08:56.679976 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.679941    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1"} err="failed to get container status \"f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1\": rpc error: code = NotFound desc = could not find container \"f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1\": container with ID starting with f32c2b4df16b52219f700ebc616244a3ccaff872043db7a8f86a9c9e22a57dc1 not found: ID does not exist"
Jul 07 15:08:56.679976 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.679953    2582 scope.go:117] "RemoveContainer" containerID="a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07"
Jul 07 15:08:56.680153 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:08:56.680122    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07\": container with ID starting with a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07 not found: ID does not exist" containerID="a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07"
Jul 07 15:08:56.680196 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.680158    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07"} err="failed to get container status \"a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07\": rpc error: code = NotFound desc = could not find container \"a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07\": container with ID starting with a4fe17e7a3e933f3fdafc72c7f3c3a25615d56ff885fa9e244389e2976e60f07 not found: ID does not exist"
Jul 07 15:08:56.680196 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.680175    2582 scope.go:117] "RemoveContainer" containerID="27bd4c7ad98005c8a114643b0db92f8ac7631b9a983046ccbe37a0fc017c7451"
Jul 07 15:08:56.680392 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:08:56.680376    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27bd4c7ad98005c8a114643b0db92f8ac7631b9a983046ccbe37a0fc017c7451\": container with ID starting with 27bd4c7ad98005c8a114643b0db92f8ac7631b9a983046ccbe37a0fc017c7451 not found: ID does not exist" containerID="27bd4c7ad98005c8a114643b0db92f8ac7631b9a983046ccbe37a0fc017c7451"
Jul 07 15:08:56.680428 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.680400    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27bd4c7ad98005c8a114643b0db92f8ac7631b9a983046ccbe37a0fc017c7451"} err="failed to get container status \"27bd4c7ad98005c8a114643b0db92f8ac7631b9a983046ccbe37a0fc017c7451\": rpc error: code = NotFound desc = could not find container \"27bd4c7ad98005c8a114643b0db92f8ac7631b9a983046ccbe37a0fc017c7451\": container with ID starting with 27bd4c7ad98005c8a114643b0db92f8ac7631b9a983046ccbe37a0fc017c7451 not found: ID does not exist"
Jul 07 15:08:56.700808 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.700783    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/713fd33d-600f-4b2d-bcc8-adb3855efe06-kserve-provision-location\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:08:56.700808 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.700810    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/713fd33d-600f-4b2d-bcc8-adb3855efe06-proxy-tls\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:08:56.700938 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.700819    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqzgl\" (UniqueName: \"kubernetes.io/projected/713fd33d-600f-4b2d-bcc8-adb3855efe06-kube-api-access-hqzgl\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:08:56.700938 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.700829    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/713fd33d-600f-4b2d-bcc8-adb3855efe06-cabundle-cert\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:08:56.700938 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.700839    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/713fd33d-600f-4b2d-bcc8-adb3855efe06-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:08:56.791910 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:56.791879    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" path="/var/lib/kubelet/pods/713fd33d-600f-4b2d-bcc8-adb3855efe06/volumes"
Jul 07 15:08:59.567796 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:59.567746    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:08:59.568233 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:08:59.568123    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:09:09.566819 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:09:09.566779    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:09:09.567340 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:09:09.567318    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:09:19.567591 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:09:19.567549    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:09:19.568151 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:09:19.567925    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:09:29.566978 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:09:29.566932    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:09:29.567416 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:09:29.567393    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:09:39.566863 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:09:39.566798    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:09:39.567303 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:09:39.567238    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:09:43.787240 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:09:43.787198    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:09:43.787621 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:09:43.787536    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:09:53.788374 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:09:53.788337    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:09:53.788823 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:09:53.788475    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:10:01.437174 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.437026    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:10:01.458641 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.458607    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"]
Jul 07 15:10:01.458941 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.458927    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="storage-initializer"
Jul 07 15:10:01.459002 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.458942    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="storage-initializer"
Jul 07 15:10:01.459002 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.458952    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent"
Jul 07 15:10:01.459002 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.458958    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent"
Jul 07 15:10:01.459002 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.458963    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kube-rbac-proxy"
Jul 07 15:10:01.459002 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.458969    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kube-rbac-proxy"
Jul 07 15:10:01.459002 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.458977    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container"
Jul 07 15:10:01.459002 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.458982    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container"
Jul 07 15:10:01.459355 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.459035    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kserve-container"
Jul 07 15:10:01.459355 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.459044    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="kube-rbac-proxy"
Jul 07 15:10:01.459355 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.459050    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="713fd33d-600f-4b2d-bcc8-adb3855efe06" containerName="agent"
Jul 07 15:10:01.463170 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.463150    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:01.466034 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.466015    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-predictor-serving-cert\""
Jul 07 15:10:01.466152 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.466059    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-kube-rbac-proxy-sar-config\""
Jul 07 15:10:01.475529 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.475504    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"]
Jul 07 15:10:01.479882 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.479858    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"]
Jul 07 15:10:01.480121 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.480100    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" containerID="cri-o://5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787" gracePeriod=30
Jul 07 15:10:01.480281 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.480189    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" containerID="cri-o://58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09" gracePeriod=30
Jul 07 15:10:01.480343 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.480248    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kube-rbac-proxy" containerID="cri-o://c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9" gracePeriod=30
Jul 07 15:10:01.550109 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.550072    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:10:01.606745 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.606711    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ea5c3a1b-4411-4120-a9d4-35a73eae0746-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-c78d7\" (UID: \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:01.606892 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.606758    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zhlq\" (UniqueName: \"kubernetes.io/projected/ea5c3a1b-4411-4120-a9d4-35a73eae0746-kube-api-access-6zhlq\") pod \"message-dumper-predictor-6575bcfdfd-c78d7\" (UID: \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:01.606892 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.606852    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ea5c3a1b-4411-4120-a9d4-35a73eae0746-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-c78d7\" (UID: \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:01.692753 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.692664    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:10:01.707829 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.707797    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zhlq\" (UniqueName: \"kubernetes.io/projected/ea5c3a1b-4411-4120-a9d4-35a73eae0746-kube-api-access-6zhlq\") pod \"message-dumper-predictor-6575bcfdfd-c78d7\" (UID: \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:01.707921 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.707849    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ea5c3a1b-4411-4120-a9d4-35a73eae0746-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-c78d7\" (UID: \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:01.707921 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.707882    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ea5c3a1b-4411-4120-a9d4-35a73eae0746-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-c78d7\" (UID: \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:01.708558 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.708533    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ea5c3a1b-4411-4120-a9d4-35a73eae0746-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-c78d7\" (UID: \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:01.710306 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.710287    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ea5c3a1b-4411-4120-a9d4-35a73eae0746-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-c78d7\" (UID: \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:01.717207 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.717182    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zhlq\" (UniqueName: \"kubernetes.io/projected/ea5c3a1b-4411-4120-a9d4-35a73eae0746-kube-api-access-6zhlq\") pod \"message-dumper-predictor-6575bcfdfd-c78d7\" (UID: \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:01.774212 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.774120    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:01.900403 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.900369    2582 generic.go:356] "Generic (PLEG): container finished" podID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerID="c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9" exitCode=2
Jul 07 15:10:01.900562 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.900441    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" event={"ID":"4848a366-b814-4b63-920a-0f3edd01a0e5","Type":"ContainerDied","Data":"c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9"}
Jul 07 15:10:01.907429 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:01.907398    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"]
Jul 07 15:10:01.909715 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:10:01.909694    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea5c3a1b_4411_4120_a9d4_35a73eae0746.slice/crio-3be7b90f456cd19e859f85d09388c999d4bec853bf1d70ec3d833ea5854e5a6c WatchSource:0}: Error finding container 3be7b90f456cd19e859f85d09388c999d4bec853bf1d70ec3d833ea5854e5a6c: Status 404 returned error can't find the container with id 3be7b90f456cd19e859f85d09388c999d4bec853bf1d70ec3d833ea5854e5a6c
Jul 07 15:10:02.904997 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:02.904962    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7" event={"ID":"ea5c3a1b-4411-4120-a9d4-35a73eae0746","Type":"ContainerStarted","Data":"3be7b90f456cd19e859f85d09388c999d4bec853bf1d70ec3d833ea5854e5a6c"}
Jul 07 15:10:03.787797 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:03.787699    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:10:03.788066 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:03.788038    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:10:03.910568 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:03.910526    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7" event={"ID":"ea5c3a1b-4411-4120-a9d4-35a73eae0746","Type":"ContainerStarted","Data":"4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783"}
Jul 07 15:10:03.910960 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:03.910630    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:03.910960 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:03.910662    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7" event={"ID":"ea5c3a1b-4411-4120-a9d4-35a73eae0746","Type":"ContainerStarted","Data":"c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4"}
Jul 07 15:10:03.910960 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:03.910707    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:03.912310 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:03.912287    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:03.933140 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:03.933082    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7" podStartSLOduration=1.3068882720000001 podStartE2EDuration="2.933069112s" totalImagesPullingTime="1.62618084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:10:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:10:03.932094514 +0000 UTC m=+807.605700570" watchObservedRunningTime="2026-07-07 15:10:03.933069112 +0000 UTC m=+807.606675168"
Jul 07 15:10:04.561658 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:04.561605    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 07 15:10:06.925031 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:06.924998    2582 generic.go:356] "Generic (PLEG): container finished" podID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerID="5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787" exitCode=0
Jul 07 15:10:06.925448 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:06.925075    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" event={"ID":"4848a366-b814-4b63-920a-0f3edd01a0e5","Type":"ContainerDied","Data":"5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787"}
Jul 07 15:10:09.561615 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:09.561567    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 07 15:10:10.924862 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:10.924830    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:10:11.483971 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.483940    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:10:11.532447 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.532403    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"]
Jul 07 15:10:11.536941 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.536913    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.541660 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.541632    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-predictor-serving-cert\""
Jul 07 15:10:11.541796 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.541638    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-kube-rbac-proxy-sar-config\""
Jul 07 15:10:11.552582 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.552560    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"]
Jul 07 15:10:11.690869 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.690830    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/142f6230-386d-460c-9876-12d4fdff3efd-proxy-tls\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.691041 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.690880    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/142f6230-386d-460c-9876-12d4fdff3efd-kserve-provision-location\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.691041 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.690975    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/142f6230-386d-460c-9876-12d4fdff3efd-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.691041 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.691014    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6g94\" (UniqueName: \"kubernetes.io/projected/142f6230-386d-460c-9876-12d4fdff3efd-kube-api-access-b6g94\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.691041 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.691039    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/142f6230-386d-460c-9876-12d4fdff3efd-cabundle-cert\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.791555 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.791462    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/142f6230-386d-460c-9876-12d4fdff3efd-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.791555 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.791515    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6g94\" (UniqueName: \"kubernetes.io/projected/142f6230-386d-460c-9876-12d4fdff3efd-kube-api-access-b6g94\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.791555 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.791547    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/142f6230-386d-460c-9876-12d4fdff3efd-cabundle-cert\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.791830 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.791594    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/142f6230-386d-460c-9876-12d4fdff3efd-proxy-tls\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.791830 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.791627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/142f6230-386d-460c-9876-12d4fdff3efd-kserve-provision-location\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.792151 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.792105    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/142f6230-386d-460c-9876-12d4fdff3efd-kserve-provision-location\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.792322 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.792298    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:10:11.792448 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.792428    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/142f6230-386d-460c-9876-12d4fdff3efd-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.792521 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.792503    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/142f6230-386d-460c-9876-12d4fdff3efd-cabundle-cert\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.794268 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.794250    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/142f6230-386d-460c-9876-12d4fdff3efd-proxy-tls\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.801145 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.801098    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6g94\" (UniqueName: \"kubernetes.io/projected/142f6230-386d-460c-9876-12d4fdff3efd-kube-api-access-b6g94\") pod \"isvc-logger-predictor-757779fc66-g2crp\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.848771 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.848735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:11.978210 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:11.978182    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"]
Jul 07 15:10:11.981230 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:10:11.981197    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142f6230_386d_460c_9876_12d4fdff3efd.slice/crio-dbcb1b7ffcd956d58b146e47f2b106480777f030aae066a5df13c23c7ff392e9 WatchSource:0}: Error finding container dbcb1b7ffcd956d58b146e47f2b106480777f030aae066a5df13c23c7ff392e9: Status 404 returned error can't find the container with id dbcb1b7ffcd956d58b146e47f2b106480777f030aae066a5df13c23c7ff392e9
Jul 07 15:10:12.950964 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:12.950931    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" event={"ID":"142f6230-386d-460c-9876-12d4fdff3efd","Type":"ContainerStarted","Data":"c228188bf9f78be57b113001d312bb05b732d6e2d75456823f97f0938f48d37f"}
Jul 07 15:10:12.951178 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:12.950993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" event={"ID":"142f6230-386d-460c-9876-12d4fdff3efd","Type":"ContainerStarted","Data":"dbcb1b7ffcd956d58b146e47f2b106480777f030aae066a5df13c23c7ff392e9"}
Jul 07 15:10:13.787221 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:13.787184    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:10:13.787655 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:13.787557    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:10:14.561657 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:14.561615    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 07 15:10:14.561849 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:14.561751    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:10:15.970232 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:15.970111    2582 generic.go:356] "Generic (PLEG): container finished" podID="142f6230-386d-460c-9876-12d4fdff3efd" containerID="c228188bf9f78be57b113001d312bb05b732d6e2d75456823f97f0938f48d37f" exitCode=0
Jul 07 15:10:15.970232 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:15.970184    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" event={"ID":"142f6230-386d-460c-9876-12d4fdff3efd","Type":"ContainerDied","Data":"c228188bf9f78be57b113001d312bb05b732d6e2d75456823f97f0938f48d37f"}
Jul 07 15:10:16.976662 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:16.976629    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" event={"ID":"142f6230-386d-460c-9876-12d4fdff3efd","Type":"ContainerStarted","Data":"520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304"}
Jul 07 15:10:16.977051 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:16.976706    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" event={"ID":"142f6230-386d-460c-9876-12d4fdff3efd","Type":"ContainerStarted","Data":"5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626"}
Jul 07 15:10:16.977051 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:16.976731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" event={"ID":"142f6230-386d-460c-9876-12d4fdff3efd","Type":"ContainerStarted","Data":"24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163"}
Jul 07 15:10:16.977051 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:16.977001    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:16.977222 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:16.977166    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:16.978359 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:16.978334    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:10:17.005220 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:17.005178    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podStartSLOduration=2.356413239 podStartE2EDuration="6.005162667s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.648749428s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:10:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:10:17.004280104 +0000 UTC m=+820.677886158" watchObservedRunningTime="2026-07-07 15:10:17.005162667 +0000 UTC m=+820.678768723"
Jul 07 15:10:17.980636 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:17.980594    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:10:17.981015 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:17.980637    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:17.981673 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:17.981650    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:10:18.984623 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:18.984581    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:10:18.985090 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:18.985065    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:10:19.561230 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:19.561191    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 07 15:10:23.787311 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:23.787271    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:5000: connect: connection refused"
Jul 07 15:10:23.787719 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:23.787425    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:10:23.787719 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:23.787655    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:10:23.787824 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:23.787756    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:10:23.989009 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:23.988979    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:10:23.989527 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:23.989488    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:10:23.989791 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:23.989766    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:10:24.560957 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:24.560917    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 07 15:10:29.560906 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:29.560866    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 07 15:10:31.635402 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.635367    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:10:31.736326 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.736230    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsds9\" (UniqueName: \"kubernetes.io/projected/4848a366-b814-4b63-920a-0f3edd01a0e5-kube-api-access-hsds9\") pod \"4848a366-b814-4b63-920a-0f3edd01a0e5\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") "
Jul 07 15:10:31.736326 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.736274    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4848a366-b814-4b63-920a-0f3edd01a0e5-kube-rbac-proxy-sar-config\") pod \"4848a366-b814-4b63-920a-0f3edd01a0e5\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") "
Jul 07 15:10:31.736326 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.736295    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4848a366-b814-4b63-920a-0f3edd01a0e5-kserve-provision-location\") pod \"4848a366-b814-4b63-920a-0f3edd01a0e5\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") "
Jul 07 15:10:31.736594 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.736335    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4848a366-b814-4b63-920a-0f3edd01a0e5-cabundle-cert\") pod \"4848a366-b814-4b63-920a-0f3edd01a0e5\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") "
Jul 07 15:10:31.736594 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.736355    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4848a366-b814-4b63-920a-0f3edd01a0e5-proxy-tls\") pod \"4848a366-b814-4b63-920a-0f3edd01a0e5\" (UID: \"4848a366-b814-4b63-920a-0f3edd01a0e5\") "
Jul 07 15:10:31.736710 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.736659    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4848a366-b814-4b63-920a-0f3edd01a0e5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4848a366-b814-4b63-920a-0f3edd01a0e5" (UID: "4848a366-b814-4b63-920a-0f3edd01a0e5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:10:31.736710 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.736683    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4848a366-b814-4b63-920a-0f3edd01a0e5-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4848a366-b814-4b63-920a-0f3edd01a0e5" (UID: "4848a366-b814-4b63-920a-0f3edd01a0e5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:10:31.736710 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.736659    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4848a366-b814-4b63-920a-0f3edd01a0e5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4848a366-b814-4b63-920a-0f3edd01a0e5" (UID: "4848a366-b814-4b63-920a-0f3edd01a0e5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:10:31.738379 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.738357    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4848a366-b814-4b63-920a-0f3edd01a0e5-kube-api-access-hsds9" (OuterVolumeSpecName: "kube-api-access-hsds9") pod "4848a366-b814-4b63-920a-0f3edd01a0e5" (UID: "4848a366-b814-4b63-920a-0f3edd01a0e5"). InnerVolumeSpecName "kube-api-access-hsds9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:10:31.738515 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.738482    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4848a366-b814-4b63-920a-0f3edd01a0e5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4848a366-b814-4b63-920a-0f3edd01a0e5" (UID: "4848a366-b814-4b63-920a-0f3edd01a0e5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:10:31.836952 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.836914    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4848a366-b814-4b63-920a-0f3edd01a0e5-cabundle-cert\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:10:31.836952 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.836944    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4848a366-b814-4b63-920a-0f3edd01a0e5-proxy-tls\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:10:31.836952 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.836957    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsds9\" (UniqueName: \"kubernetes.io/projected/4848a366-b814-4b63-920a-0f3edd01a0e5-kube-api-access-hsds9\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:10:31.837223 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.836971    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4848a366-b814-4b63-920a-0f3edd01a0e5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:10:31.837223 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:31.836983    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4848a366-b814-4b63-920a-0f3edd01a0e5-kserve-provision-location\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:10:32.039787 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.039684    2582 generic.go:356] "Generic (PLEG): container finished" podID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerID="58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09" exitCode=0
Jul 07 15:10:32.039787 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.039738    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" event={"ID":"4848a366-b814-4b63-920a-0f3edd01a0e5","Type":"ContainerDied","Data":"58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09"}
Jul 07 15:10:32.039787 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.039752    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"
Jul 07 15:10:32.040116 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.039805    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p" event={"ID":"4848a366-b814-4b63-920a-0f3edd01a0e5","Type":"ContainerDied","Data":"65aabfb4ffa524b48cb020d16aecfae1f789301b16ced88808ee438498da0061"}
Jul 07 15:10:32.040116 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.039838    2582 scope.go:117] "RemoveContainer" containerID="58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09"
Jul 07 15:10:32.053033 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.053011    2582 scope.go:117] "RemoveContainer" containerID="c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9"
Jul 07 15:10:32.060463 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.060433    2582 scope.go:117] "RemoveContainer" containerID="5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787"
Jul 07 15:10:32.067372 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.067356    2582 scope.go:117] "RemoveContainer" containerID="a21ed028dc6bc2684ccd979c6eab4094268fd006a5cc8698f21a7f31a2919d10"
Jul 07 15:10:32.070466 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.070444    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"]
Jul 07 15:10:32.075238 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.075218    2582 scope.go:117] "RemoveContainer" containerID="58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09"
Jul 07 15:10:32.075342 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.075303    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-684888dfc6-bm24p"]
Jul 07 15:10:32.075510 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:10:32.075484    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09\": container with ID starting with 58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09 not found: ID does not exist" containerID="58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09"
Jul 07 15:10:32.075550 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.075520    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09"} err="failed to get container status \"58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09\": rpc error: code = NotFound desc = could not find container \"58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09\": container with ID starting with 58f2d036a629218df69ff872b6438f4e63901fd5a6647e3f1fb51f7a8b308e09 not found: ID does not exist"
Jul 07 15:10:32.075550 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.075540    2582 scope.go:117] "RemoveContainer" containerID="c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9"
Jul 07 15:10:32.075806 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:10:32.075788    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9\": container with ID starting with c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9 not found: ID does not exist" containerID="c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9"
Jul 07 15:10:32.075850 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.075812    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9"} err="failed to get container status \"c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9\": rpc error: code = NotFound desc = could not find container \"c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9\": container with ID starting with c426c8a3671993e5e9ff37a4c6023bdbd6a61507aede6a42fca56ef141e79fa9 not found: ID does not exist"
Jul 07 15:10:32.075850 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.075827    2582 scope.go:117] "RemoveContainer" containerID="5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787"
Jul 07 15:10:32.076028 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:10:32.076013    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787\": container with ID starting with 5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787 not found: ID does not exist" containerID="5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787"
Jul 07 15:10:32.076067 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.076033    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787"} err="failed to get container status \"5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787\": rpc error: code = NotFound desc = could not find container \"5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787\": container with ID starting with 5f8beae339c652b0a5e21c1b027f954aecf0342c0cd1c64cd213a37c61c96787 not found: ID does not exist"
Jul 07 15:10:32.076067 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.076046    2582 scope.go:117] "RemoveContainer" containerID="a21ed028dc6bc2684ccd979c6eab4094268fd006a5cc8698f21a7f31a2919d10"
Jul 07 15:10:32.076248 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:10:32.076233    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a21ed028dc6bc2684ccd979c6eab4094268fd006a5cc8698f21a7f31a2919d10\": container with ID starting with a21ed028dc6bc2684ccd979c6eab4094268fd006a5cc8698f21a7f31a2919d10 not found: ID does not exist" containerID="a21ed028dc6bc2684ccd979c6eab4094268fd006a5cc8698f21a7f31a2919d10"
Jul 07 15:10:32.076288 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.076250    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a21ed028dc6bc2684ccd979c6eab4094268fd006a5cc8698f21a7f31a2919d10"} err="failed to get container status \"a21ed028dc6bc2684ccd979c6eab4094268fd006a5cc8698f21a7f31a2919d10\": rpc error: code = NotFound desc = could not find container \"a21ed028dc6bc2684ccd979c6eab4094268fd006a5cc8698f21a7f31a2919d10\": container with ID starting with a21ed028dc6bc2684ccd979c6eab4094268fd006a5cc8698f21a7f31a2919d10 not found: ID does not exist"
Jul 07 15:10:32.792614 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:32.792583    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" path="/var/lib/kubelet/pods/4848a366-b814-4b63-920a-0f3edd01a0e5/volumes"
Jul 07 15:10:33.989939 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:33.989900    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:10:33.990386 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:33.990315    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:10:43.989464 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:43.989419    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:10:43.989890 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:43.989849    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:10:53.990319 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:53.990276    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:10:53.990784 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:10:53.990683    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:11:03.989849 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:03.989802    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:11:03.990415 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:03.990164    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:11:13.990094 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:13.990050    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:11:13.990551 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:13.990533    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:11:23.646535 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:23.646501    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:11:23.990063 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:23.990020    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:11:23.990534 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:23.990510    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:11:33.990689 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:33.990654    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:11:33.991083 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:33.990847    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:11:46.495457 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:46.495421    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-predictor-6575bcfdfd-c78d7_ea5c3a1b-4411-4120-a9d4-35a73eae0746/kserve-container/0.log"
Jul 07 15:11:46.749162 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:46.749072    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:11:46.825609 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:46.825579    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"]
Jul 07 15:11:46.825926 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:46.825899    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" containerID="cri-o://24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163" gracePeriod=30
Jul 07 15:11:46.826123 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:46.826067    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" containerID="cri-o://520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304" gracePeriod=30
Jul 07 15:11:46.826292 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:46.826240    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kube-rbac-proxy" containerID="cri-o://5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626" gracePeriod=30
Jul 07 15:11:46.847996 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:46.847949    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:11:46.870805 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:46.870778    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"]
Jul 07 15:11:46.871052 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:46.871027    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7" podUID="ea5c3a1b-4411-4120-a9d4-35a73eae0746" containerName="kube-rbac-proxy" containerID="cri-o://4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783" gracePeriod=30
Jul 07 15:11:46.871052 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:46.871019    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7" podUID="ea5c3a1b-4411-4120-a9d4-35a73eae0746" containerName="kserve-container" containerID="cri-o://c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4" gracePeriod=30
Jul 07 15:11:46.950171 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:46.950142    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:11:47.056065 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.056034    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:11:47.113193 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.113163    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:11:47.141700 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.141674    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zhlq\" (UniqueName: \"kubernetes.io/projected/ea5c3a1b-4411-4120-a9d4-35a73eae0746-kube-api-access-6zhlq\") pod \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\" (UID: \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\") "
Jul 07 15:11:47.141813 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.141750    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ea5c3a1b-4411-4120-a9d4-35a73eae0746-kube-rbac-proxy-sar-config\") pod \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\" (UID: \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\") "
Jul 07 15:11:47.141813 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.141790    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ea5c3a1b-4411-4120-a9d4-35a73eae0746-proxy-tls\") pod \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\" (UID: \"ea5c3a1b-4411-4120-a9d4-35a73eae0746\") "
Jul 07 15:11:47.142150 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.142103    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea5c3a1b-4411-4120-a9d4-35a73eae0746-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ea5c3a1b-4411-4120-a9d4-35a73eae0746" (UID: "ea5c3a1b-4411-4120-a9d4-35a73eae0746"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:11:47.143862 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.143833    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea5c3a1b-4411-4120-a9d4-35a73eae0746-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ea5c3a1b-4411-4120-a9d4-35a73eae0746" (UID: "ea5c3a1b-4411-4120-a9d4-35a73eae0746"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:11:47.143862 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.143851    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea5c3a1b-4411-4120-a9d4-35a73eae0746-kube-api-access-6zhlq" (OuterVolumeSpecName: "kube-api-access-6zhlq") pod "ea5c3a1b-4411-4120-a9d4-35a73eae0746" (UID: "ea5c3a1b-4411-4120-a9d4-35a73eae0746"). InnerVolumeSpecName "kube-api-access-6zhlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:11:47.242502 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.242468    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ea5c3a1b-4411-4120-a9d4-35a73eae0746-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:11:47.242502 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.242496    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ea5c3a1b-4411-4120-a9d4-35a73eae0746-proxy-tls\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:11:47.242502 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.242510    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zhlq\" (UniqueName: \"kubernetes.io/projected/ea5c3a1b-4411-4120-a9d4-35a73eae0746-kube-api-access-6zhlq\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:11:47.328004 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.327920    2582 generic.go:356] "Generic (PLEG): container finished" podID="ea5c3a1b-4411-4120-a9d4-35a73eae0746" containerID="4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783" exitCode=2
Jul 07 15:11:47.328004 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.327946    2582 generic.go:356] "Generic (PLEG): container finished" podID="ea5c3a1b-4411-4120-a9d4-35a73eae0746" containerID="c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4" exitCode=2
Jul 07 15:11:47.328004 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.327965    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"
Jul 07 15:11:47.328319 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.328011    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7" event={"ID":"ea5c3a1b-4411-4120-a9d4-35a73eae0746","Type":"ContainerDied","Data":"4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783"}
Jul 07 15:11:47.328319 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.328081    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7" event={"ID":"ea5c3a1b-4411-4120-a9d4-35a73eae0746","Type":"ContainerDied","Data":"c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4"}
Jul 07 15:11:47.328319 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.328112    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7" event={"ID":"ea5c3a1b-4411-4120-a9d4-35a73eae0746","Type":"ContainerDied","Data":"3be7b90f456cd19e859f85d09388c999d4bec853bf1d70ec3d833ea5854e5a6c"}
Jul 07 15:11:47.328319 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.328173    2582 scope.go:117] "RemoveContainer" containerID="4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783"
Jul 07 15:11:47.330839 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.330818    2582 generic.go:356] "Generic (PLEG): container finished" podID="142f6230-386d-460c-9876-12d4fdff3efd" containerID="5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626" exitCode=2
Jul 07 15:11:47.330935 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.330845    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" event={"ID":"142f6230-386d-460c-9876-12d4fdff3efd","Type":"ContainerDied","Data":"5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626"}
Jul 07 15:11:47.336821 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.336802    2582 scope.go:117] "RemoveContainer" containerID="c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4"
Jul 07 15:11:47.343833 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.343813    2582 scope.go:117] "RemoveContainer" containerID="4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783"
Jul 07 15:11:47.344095 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:11:47.344076    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783\": container with ID starting with 4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783 not found: ID does not exist" containerID="4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783"
Jul 07 15:11:47.344189 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.344103    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783"} err="failed to get container status \"4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783\": rpc error: code = NotFound desc = could not find container \"4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783\": container with ID starting with 4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783 not found: ID does not exist"
Jul 07 15:11:47.344189 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.344119    2582 scope.go:117] "RemoveContainer" containerID="c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4"
Jul 07 15:11:47.344366 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:11:47.344348    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4\": container with ID starting with c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4 not found: ID does not exist" containerID="c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4"
Jul 07 15:11:47.344426 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.344375    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4"} err="failed to get container status \"c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4\": rpc error: code = NotFound desc = could not find container \"c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4\": container with ID starting with c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4 not found: ID does not exist"
Jul 07 15:11:47.344426 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.344397    2582 scope.go:117] "RemoveContainer" containerID="4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783"
Jul 07 15:11:47.344622 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.344600    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783"} err="failed to get container status \"4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783\": rpc error: code = NotFound desc = could not find container \"4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783\": container with ID starting with 4607b132e0988f9ffb2cdccdc58a4668e774e485dc6e622f79c38f1464dd0783 not found: ID does not exist"
Jul 07 15:11:47.344678 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.344623    2582 scope.go:117] "RemoveContainer" containerID="c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4"
Jul 07 15:11:47.344807 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.344787    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4"} err="failed to get container status \"c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4\": rpc error: code = NotFound desc = could not find container \"c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4\": container with ID starting with c1923d4fb159b041ca686234448e94d5e6a72a505c3171d79631831be5a017d4 not found: ID does not exist"
Jul 07 15:11:47.350880 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.350858    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"]
Jul 07 15:11:47.352848 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:47.352823    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-c78d7"]
Jul 07 15:11:48.791806 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:48.791770    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea5c3a1b-4411-4120-a9d4-35a73eae0746" path="/var/lib/kubelet/pods/ea5c3a1b-4411-4120-a9d4-35a73eae0746/volumes"
Jul 07 15:11:48.985833 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:48.985789    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Jul 07 15:11:51.355677 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:51.355642    2582 generic.go:356] "Generic (PLEG): container finished" podID="142f6230-386d-460c-9876-12d4fdff3efd" containerID="24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163" exitCode=0
Jul 07 15:11:51.356039 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:51.355682    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" event={"ID":"142f6230-386d-460c-9876-12d4fdff3efd","Type":"ContainerDied","Data":"24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163"}
Jul 07 15:11:53.985845 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:53.985802    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Jul 07 15:11:53.990177 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:53.990141    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:11:53.990560 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:53.990536    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:11:58.985829 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:58.985780    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Jul 07 15:11:58.986313 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:11:58.985911    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:12:03.985743 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:03.985697    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Jul 07 15:12:03.989993 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:03.989963    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:12:03.990293 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:03.990267    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:12:08.985229 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:08.985188    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Jul 07 15:12:13.985554 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:13.985508    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Jul 07 15:12:13.989831 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:13.989799    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 07 15:12:13.989968 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:13.989952    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:12:13.990174 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:13.990157    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:12:13.990248 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:13.990232    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:12:16.979417 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:16.979392    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:12:17.073373 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.073333    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/142f6230-386d-460c-9876-12d4fdff3efd-kube-rbac-proxy-sar-config\") pod \"142f6230-386d-460c-9876-12d4fdff3efd\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") "
Jul 07 15:12:17.073559 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.073395    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/142f6230-386d-460c-9876-12d4fdff3efd-kserve-provision-location\") pod \"142f6230-386d-460c-9876-12d4fdff3efd\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") "
Jul 07 15:12:17.073559 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.073439    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/142f6230-386d-460c-9876-12d4fdff3efd-proxy-tls\") pod \"142f6230-386d-460c-9876-12d4fdff3efd\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") "
Jul 07 15:12:17.073559 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.073461    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/142f6230-386d-460c-9876-12d4fdff3efd-cabundle-cert\") pod \"142f6230-386d-460c-9876-12d4fdff3efd\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") "
Jul 07 15:12:17.073559 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.073528    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6g94\" (UniqueName: \"kubernetes.io/projected/142f6230-386d-460c-9876-12d4fdff3efd-kube-api-access-b6g94\") pod \"142f6230-386d-460c-9876-12d4fdff3efd\" (UID: \"142f6230-386d-460c-9876-12d4fdff3efd\") "
Jul 07 15:12:17.073815 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.073783    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142f6230-386d-460c-9876-12d4fdff3efd-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "142f6230-386d-460c-9876-12d4fdff3efd" (UID: "142f6230-386d-460c-9876-12d4fdff3efd"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:12:17.073875 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.073786    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/142f6230-386d-460c-9876-12d4fdff3efd-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "142f6230-386d-460c-9876-12d4fdff3efd" (UID: "142f6230-386d-460c-9876-12d4fdff3efd"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:12:17.073875 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.073862    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/142f6230-386d-460c-9876-12d4fdff3efd-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "142f6230-386d-460c-9876-12d4fdff3efd" (UID: "142f6230-386d-460c-9876-12d4fdff3efd"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:12:17.075542 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.075509    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/142f6230-386d-460c-9876-12d4fdff3efd-kube-api-access-b6g94" (OuterVolumeSpecName: "kube-api-access-b6g94") pod "142f6230-386d-460c-9876-12d4fdff3efd" (UID: "142f6230-386d-460c-9876-12d4fdff3efd"). InnerVolumeSpecName "kube-api-access-b6g94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:12:17.075650 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.075625    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/142f6230-386d-460c-9876-12d4fdff3efd-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "142f6230-386d-460c-9876-12d4fdff3efd" (UID: "142f6230-386d-460c-9876-12d4fdff3efd"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:12:17.175026 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.174947    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/142f6230-386d-460c-9876-12d4fdff3efd-proxy-tls\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:12:17.175026 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.174972    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/142f6230-386d-460c-9876-12d4fdff3efd-cabundle-cert\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:12:17.175026 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.174981    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6g94\" (UniqueName: \"kubernetes.io/projected/142f6230-386d-460c-9876-12d4fdff3efd-kube-api-access-b6g94\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:12:17.175026 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.174990    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/142f6230-386d-460c-9876-12d4fdff3efd-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:12:17.175026 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.175002    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/142f6230-386d-460c-9876-12d4fdff3efd-kserve-provision-location\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:12:17.451181 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.451081    2582 generic.go:356] "Generic (PLEG): container finished" podID="142f6230-386d-460c-9876-12d4fdff3efd" containerID="520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304" exitCode=0
Jul 07 15:12:17.451181 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.451152    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" event={"ID":"142f6230-386d-460c-9876-12d4fdff3efd","Type":"ContainerDied","Data":"520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304"}
Jul 07 15:12:17.451181 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.451163    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"
Jul 07 15:12:17.451411 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.451205    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp" event={"ID":"142f6230-386d-460c-9876-12d4fdff3efd","Type":"ContainerDied","Data":"dbcb1b7ffcd956d58b146e47f2b106480777f030aae066a5df13c23c7ff392e9"}
Jul 07 15:12:17.451411 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.451239    2582 scope.go:117] "RemoveContainer" containerID="520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304"
Jul 07 15:12:17.459581 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.459566    2582 scope.go:117] "RemoveContainer" containerID="5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626"
Jul 07 15:12:17.466576 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.466561    2582 scope.go:117] "RemoveContainer" containerID="24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163"
Jul 07 15:12:17.473193 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.473173    2582 scope.go:117] "RemoveContainer" containerID="c228188bf9f78be57b113001d312bb05b732d6e2d75456823f97f0938f48d37f"
Jul 07 15:12:17.478369 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.478350    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"]
Jul 07 15:12:17.480265 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.480246    2582 scope.go:117] "RemoveContainer" containerID="520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304"
Jul 07 15:12:17.480545 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:12:17.480524    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304\": container with ID starting with 520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304 not found: ID does not exist" containerID="520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304"
Jul 07 15:12:17.480631 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.480557    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304"} err="failed to get container status \"520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304\": rpc error: code = NotFound desc = could not find container \"520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304\": container with ID starting with 520139cf9ed9444846e66df94d3a81c0f40cb2bd07b36cadc486cf7b9e885304 not found: ID does not exist"
Jul 07 15:12:17.480631 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.480582    2582 scope.go:117] "RemoveContainer" containerID="5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626"
Jul 07 15:12:17.480851 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:12:17.480834    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626\": container with ID starting with 5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626 not found: ID does not exist" containerID="5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626"
Jul 07 15:12:17.480889 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.480855    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626"} err="failed to get container status \"5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626\": rpc error: code = NotFound desc = could not find container \"5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626\": container with ID starting with 5097bd9e9ae91d1d851cfb02fb4a418e0dc1a1d9407688d97751b88160cc9626 not found: ID does not exist"
Jul 07 15:12:17.480889 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.480870    2582 scope.go:117] "RemoveContainer" containerID="24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163"
Jul 07 15:12:17.481090 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:12:17.481072    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163\": container with ID starting with 24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163 not found: ID does not exist" containerID="24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163"
Jul 07 15:12:17.481153 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.481097    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163"} err="failed to get container status \"24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163\": rpc error: code = NotFound desc = could not find container \"24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163\": container with ID starting with 24840f698f4787b90b5c4b3a2980899bb97ba4403719a6f8e0a542774a439163 not found: ID does not exist"
Jul 07 15:12:17.481153 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.481114    2582 scope.go:117] "RemoveContainer" containerID="c228188bf9f78be57b113001d312bb05b732d6e2d75456823f97f0938f48d37f"
Jul 07 15:12:17.481370 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:12:17.481354    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c228188bf9f78be57b113001d312bb05b732d6e2d75456823f97f0938f48d37f\": container with ID starting with c228188bf9f78be57b113001d312bb05b732d6e2d75456823f97f0938f48d37f not found: ID does not exist" containerID="c228188bf9f78be57b113001d312bb05b732d6e2d75456823f97f0938f48d37f"
Jul 07 15:12:17.481423 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.481377    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c228188bf9f78be57b113001d312bb05b732d6e2d75456823f97f0938f48d37f"} err="failed to get container status \"c228188bf9f78be57b113001d312bb05b732d6e2d75456823f97f0938f48d37f\": rpc error: code = NotFound desc = could not find container \"c228188bf9f78be57b113001d312bb05b732d6e2d75456823f97f0938f48d37f\": container with ID starting with c228188bf9f78be57b113001d312bb05b732d6e2d75456823f97f0938f48d37f not found: ID does not exist"
Jul 07 15:12:17.485107 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:17.485087    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-757779fc66-g2crp"]
Jul 07 15:12:18.791677 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:18.791647    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="142f6230-386d-460c-9876-12d4fdff3efd" path="/var/lib/kubelet/pods/142f6230-386d-460c-9876-12d4fdff3efd/volumes"
Jul 07 15:12:50.185221 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:12:50.185180    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:14:15.484108 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:14:15.484070    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:15:06.978019 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:06.977981    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:15:12.213301 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.213243    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:15:12.269178 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269141    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"]
Jul 07 15:15:12.269445 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269431    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent"
Jul 07 15:15:12.269489 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269447    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent"
Jul 07 15:15:12.269489 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269459    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="storage-initializer"
Jul 07 15:15:12.269489 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269466    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="storage-initializer"
Jul 07 15:15:12.269489 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269484    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container"
Jul 07 15:15:12.269489 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269490    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269498    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kube-rbac-proxy"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269503    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kube-rbac-proxy"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269509    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea5c3a1b-4411-4120-a9d4-35a73eae0746" containerName="kserve-container"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269514    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea5c3a1b-4411-4120-a9d4-35a73eae0746" containerName="kserve-container"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269520    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kube-rbac-proxy"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269525    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kube-rbac-proxy"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269532    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269537    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269545    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269554    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269562    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea5c3a1b-4411-4120-a9d4-35a73eae0746" containerName="kube-rbac-proxy"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269567    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea5c3a1b-4411-4120-a9d4-35a73eae0746" containerName="kube-rbac-proxy"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269574    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="storage-initializer"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269579    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="storage-initializer"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269623    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kube-rbac-proxy"
Jul 07 15:15:12.269635 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269637    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea5c3a1b-4411-4120-a9d4-35a73eae0746" containerName="kserve-container"
Jul 07 15:15:12.270113 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269645    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="kserve-container"
Jul 07 15:15:12.270113 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269651    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4848a366-b814-4b63-920a-0f3edd01a0e5" containerName="agent"
Jul 07 15:15:12.270113 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269656    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea5c3a1b-4411-4120-a9d4-35a73eae0746" containerName="kube-rbac-proxy"
Jul 07 15:15:12.270113 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269661    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kube-rbac-proxy"
Jul 07 15:15:12.270113 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269666    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="agent"
Jul 07 15:15:12.270113 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.269675    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="142f6230-386d-460c-9876-12d4fdff3efd" containerName="kserve-container"
Jul 07 15:15:12.272849 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.272830    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.275750 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.275725    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-predictor-serving-cert\""
Jul 07 15:15:12.275881 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.275758    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 07 15:15:12.276227 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.276211    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 07 15:15:12.276983 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.276962    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 07 15:15:12.277087 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.276965    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-bnv6j\""
Jul 07 15:15:12.277087 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.277010    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 07 15:15:12.277087 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.277014    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-kube-rbac-proxy-sar-config\""
Jul 07 15:15:12.284410 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.284386    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"]
Jul 07 15:15:12.386481 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.386432    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d731d089-94f4-4860-a873-f9bcd2c97169-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.386669 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.386518    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wqg9\" (UniqueName: \"kubernetes.io/projected/d731d089-94f4-4860-a873-f9bcd2c97169-kube-api-access-9wqg9\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.386669 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.386565    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d731d089-94f4-4860-a873-f9bcd2c97169-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.386669 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.386624    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d731d089-94f4-4860-a873-f9bcd2c97169-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.386811 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.386686    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d731d089-94f4-4860-a873-f9bcd2c97169-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.487366 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.487329    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d731d089-94f4-4860-a873-f9bcd2c97169-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.487553 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.487381    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d731d089-94f4-4860-a873-f9bcd2c97169-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.487553 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.487417    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d731d089-94f4-4860-a873-f9bcd2c97169-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.487553 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.487449    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wqg9\" (UniqueName: \"kubernetes.io/projected/d731d089-94f4-4860-a873-f9bcd2c97169-kube-api-access-9wqg9\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.487553 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.487482    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d731d089-94f4-4860-a873-f9bcd2c97169-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.487763 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:15:12.487744    2582 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v2-predictor-serving-cert: secret "isvc-autogluon-v2-predictor-serving-cert" not found
Jul 07 15:15:12.487819 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.487782    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d731d089-94f4-4860-a873-f9bcd2c97169-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.487871 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:15:12.487836    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d731d089-94f4-4860-a873-f9bcd2c97169-proxy-tls podName:d731d089-94f4-4860-a873-f9bcd2c97169 nodeName:}" failed. No retries permitted until 2026-07-07 15:15:12.987811977 +0000 UTC m=+1116.661418026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d731d089-94f4-4860-a873-f9bcd2c97169-proxy-tls") pod "isvc-autogluon-v2-predictor-67b849d456-8p5bm" (UID: "d731d089-94f4-4860-a873-f9bcd2c97169") : secret "isvc-autogluon-v2-predictor-serving-cert" not found
Jul 07 15:15:12.488142 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.488108    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d731d089-94f4-4860-a873-f9bcd2c97169-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.488208 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.488166    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d731d089-94f4-4860-a873-f9bcd2c97169-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.500892 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.500853    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wqg9\" (UniqueName: \"kubernetes.io/projected/d731d089-94f4-4860-a873-f9bcd2c97169-kube-api-access-9wqg9\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.606787 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.606758    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:15:12.992113 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.992076    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d731d089-94f4-4860-a873-f9bcd2c97169-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:12.994483 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:12.994461    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d731d089-94f4-4860-a873-f9bcd2c97169-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-8p5bm\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:13.183817 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:13.183773    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:15:13.313623 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:13.313590    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"]
Jul 07 15:15:13.317153 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:15:13.317099    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd731d089_94f4_4860_a873_f9bcd2c97169.slice/crio-fd991ea885d65ec82151f2748fae6a58ea48fcf36dbbc3cbbbf94cff096f63b7 WatchSource:0}: Error finding container fd991ea885d65ec82151f2748fae6a58ea48fcf36dbbc3cbbbf94cff096f63b7: Status 404 returned error can't find the container with id fd991ea885d65ec82151f2748fae6a58ea48fcf36dbbc3cbbbf94cff096f63b7
Jul 07 15:15:14.070142 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:14.070105    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm" event={"ID":"d731d089-94f4-4860-a873-f9bcd2c97169","Type":"ContainerStarted","Data":"270db3673fa424ac966b0a1939d223b7bba7e54b66b464b91eb1d97141170243"}
Jul 07 15:15:14.070316 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:14.070185    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm" event={"ID":"d731d089-94f4-4860-a873-f9bcd2c97169","Type":"ContainerStarted","Data":"fd991ea885d65ec82151f2748fae6a58ea48fcf36dbbc3cbbbf94cff096f63b7"}
Jul 07 15:15:43.178220 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:43.178186    2582 generic.go:356] "Generic (PLEG): container finished" podID="d731d089-94f4-4860-a873-f9bcd2c97169" containerID="270db3673fa424ac966b0a1939d223b7bba7e54b66b464b91eb1d97141170243" exitCode=0
Jul 07 15:15:43.178724 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:43.178269    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm" event={"ID":"d731d089-94f4-4860-a873-f9bcd2c97169","Type":"ContainerDied","Data":"270db3673fa424ac966b0a1939d223b7bba7e54b66b464b91eb1d97141170243"}
Jul 07 15:15:43.179678 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:15:43.179664    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:16:16.075883 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:16:16.075811    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:18:09.769394 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:18:09.769359    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm" event={"ID":"d731d089-94f4-4860-a873-f9bcd2c97169","Type":"ContainerStarted","Data":"bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19"}
Jul 07 15:18:10.775058 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:18:10.775022    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm" event={"ID":"d731d089-94f4-4860-a873-f9bcd2c97169","Type":"ContainerStarted","Data":"be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915"}
Jul 07 15:18:10.775558 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:18:10.775168    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:18:10.812825 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:18:10.812769    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm" podStartSLOduration=3.07416589 podStartE2EDuration="2m58.812753593s" totalImagesPullingTime="2m26.460753018s" totalInitContainerRuntime="29.277834685s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:15:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:18:10.81163784 +0000 UTC m=+1294.485243895" watchObservedRunningTime="2026-07-07 15:18:10.812753593 +0000 UTC m=+1294.486359648"
Jul 07 15:18:11.779098 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:18:11.779059    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:18:14.000300 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:18:14.000267    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:18:17.788264 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:18:17.788234    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:19:17.931938 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:17.931860    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:19:18.029780 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:18.029751    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:19:44.791899 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:44.791870    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:19:52.612975 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:52.612936    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"]
Jul 07 15:19:52.613553 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:52.613199    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm" podUID="d731d089-94f4-4860-a873-f9bcd2c97169" containerName="kserve-container" containerID="cri-o://bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19" gracePeriod=30
Jul 07 15:19:52.613553 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:52.613252    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm" podUID="d731d089-94f4-4860-a873-f9bcd2c97169" containerName="kube-rbac-proxy" containerID="cri-o://be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915" gracePeriod=30
Jul 07 15:19:52.617420 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:52.617394    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:19:52.783338 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:52.783283    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm" podUID="d731d089-94f4-4860-a873-f9bcd2c97169" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 07 15:19:53.150514 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.150478    2582 generic.go:356] "Generic (PLEG): container finished" podID="d731d089-94f4-4860-a873-f9bcd2c97169" containerID="be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915" exitCode=2
Jul 07 15:19:53.150706 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.150551    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm" event={"ID":"d731d089-94f4-4860-a873-f9bcd2c97169","Type":"ContainerDied","Data":"be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915"}
Jul 07 15:19:53.456706 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.456685    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:19:53.555601 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.555563    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d731d089-94f4-4860-a873-f9bcd2c97169-kserve-provision-location\") pod \"d731d089-94f4-4860-a873-f9bcd2c97169\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") "
Jul 07 15:19:53.555797 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.555637    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wqg9\" (UniqueName: \"kubernetes.io/projected/d731d089-94f4-4860-a873-f9bcd2c97169-kube-api-access-9wqg9\") pod \"d731d089-94f4-4860-a873-f9bcd2c97169\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") "
Jul 07 15:19:53.555797 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.555661    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d731d089-94f4-4860-a873-f9bcd2c97169-cabundle-cert\") pod \"d731d089-94f4-4860-a873-f9bcd2c97169\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") "
Jul 07 15:19:53.555797 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.555695    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d731d089-94f4-4860-a873-f9bcd2c97169-proxy-tls\") pod \"d731d089-94f4-4860-a873-f9bcd2c97169\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") "
Jul 07 15:19:53.555797 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.555735    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d731d089-94f4-4860-a873-f9bcd2c97169-kube-rbac-proxy-sar-config\") pod \"d731d089-94f4-4860-a873-f9bcd2c97169\" (UID: \"d731d089-94f4-4860-a873-f9bcd2c97169\") "
Jul 07 15:19:53.556107 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.556071    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d731d089-94f4-4860-a873-f9bcd2c97169-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "d731d089-94f4-4860-a873-f9bcd2c97169" (UID: "d731d089-94f4-4860-a873-f9bcd2c97169"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:19:53.556256 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.556122    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d731d089-94f4-4860-a873-f9bcd2c97169-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d731d089-94f4-4860-a873-f9bcd2c97169" (UID: "d731d089-94f4-4860-a873-f9bcd2c97169"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:19:53.557878 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.557860    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d731d089-94f4-4860-a873-f9bcd2c97169-kube-api-access-9wqg9" (OuterVolumeSpecName: "kube-api-access-9wqg9") pod "d731d089-94f4-4860-a873-f9bcd2c97169" (UID: "d731d089-94f4-4860-a873-f9bcd2c97169"). InnerVolumeSpecName "kube-api-access-9wqg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:19:53.557878 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.557862    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d731d089-94f4-4860-a873-f9bcd2c97169-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d731d089-94f4-4860-a873-f9bcd2c97169" (UID: "d731d089-94f4-4860-a873-f9bcd2c97169"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:19:53.560328 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.560305    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d731d089-94f4-4860-a873-f9bcd2c97169-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d731d089-94f4-4860-a873-f9bcd2c97169" (UID: "d731d089-94f4-4860-a873-f9bcd2c97169"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:19:53.656851 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.656816    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wqg9\" (UniqueName: \"kubernetes.io/projected/d731d089-94f4-4860-a873-f9bcd2c97169-kube-api-access-9wqg9\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:19:53.656851 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.656845    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d731d089-94f4-4860-a873-f9bcd2c97169-cabundle-cert\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:19:53.656851 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.656856    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d731d089-94f4-4860-a873-f9bcd2c97169-proxy-tls\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:19:53.657312 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.656864    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d731d089-94f4-4860-a873-f9bcd2c97169-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:19:53.657312 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:53.656873    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d731d089-94f4-4860-a873-f9bcd2c97169-kserve-provision-location\") on node \"ip-10-0-140-158.ec2.internal\" DevicePath \"\""
Jul 07 15:19:54.155956 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.155921    2582 generic.go:356] "Generic (PLEG): container finished" podID="d731d089-94f4-4860-a873-f9bcd2c97169" containerID="bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19" exitCode=0
Jul 07 15:19:54.156167 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.155985    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"
Jul 07 15:19:54.156167 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.156005    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm" event={"ID":"d731d089-94f4-4860-a873-f9bcd2c97169","Type":"ContainerDied","Data":"bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19"}
Jul 07 15:19:54.156167 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.156068    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm" event={"ID":"d731d089-94f4-4860-a873-f9bcd2c97169","Type":"ContainerDied","Data":"fd991ea885d65ec82151f2748fae6a58ea48fcf36dbbc3cbbbf94cff096f63b7"}
Jul 07 15:19:54.156167 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.156094    2582 scope.go:117] "RemoveContainer" containerID="be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915"
Jul 07 15:19:54.164714 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.164671    2582 scope.go:117] "RemoveContainer" containerID="bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19"
Jul 07 15:19:54.171563 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.171543    2582 scope.go:117] "RemoveContainer" containerID="270db3673fa424ac966b0a1939d223b7bba7e54b66b464b91eb1d97141170243"
Jul 07 15:19:54.178704 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.178670    2582 scope.go:117] "RemoveContainer" containerID="be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915"
Jul 07 15:19:54.178958 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:19:54.178940    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915\": container with ID starting with be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915 not found: ID does not exist" containerID="be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915"
Jul 07 15:19:54.179003 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.178968    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915"} err="failed to get container status \"be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915\": rpc error: code = NotFound desc = could not find container \"be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915\": container with ID starting with be2d2d039e496c6160a9765bc4932643df7661214652c4ae7eeaa5cda735a915 not found: ID does not exist"
Jul 07 15:19:54.179003 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.178989    2582 scope.go:117] "RemoveContainer" containerID="bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19"
Jul 07 15:19:54.179280 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:19:54.179262    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19\": container with ID starting with bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19 not found: ID does not exist" containerID="bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19"
Jul 07 15:19:54.179359 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.179284    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19"} err="failed to get container status \"bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19\": rpc error: code = NotFound desc = could not find container \"bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19\": container with ID starting with bd089bafa02c15e113b036daf95e18b3b0d9fcab22b3b32a36901651f5356d19 not found: ID does not exist"
Jul 07 15:19:54.179359 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.179303    2582 scope.go:117] "RemoveContainer" containerID="270db3673fa424ac966b0a1939d223b7bba7e54b66b464b91eb1d97141170243"
Jul 07 15:19:54.179537 ip-10-0-140-158 kubenswrapper[2582]: E0707 15:19:54.179517    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"270db3673fa424ac966b0a1939d223b7bba7e54b66b464b91eb1d97141170243\": container with ID starting with 270db3673fa424ac966b0a1939d223b7bba7e54b66b464b91eb1d97141170243 not found: ID does not exist" containerID="270db3673fa424ac966b0a1939d223b7bba7e54b66b464b91eb1d97141170243"
Jul 07 15:19:54.179597 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.179542    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"270db3673fa424ac966b0a1939d223b7bba7e54b66b464b91eb1d97141170243"} err="failed to get container status \"270db3673fa424ac966b0a1939d223b7bba7e54b66b464b91eb1d97141170243\": rpc error: code = NotFound desc = could not find container \"270db3673fa424ac966b0a1939d223b7bba7e54b66b464b91eb1d97141170243\": container with ID starting with 270db3673fa424ac966b0a1939d223b7bba7e54b66b464b91eb1d97141170243 not found: ID does not exist"
Jul 07 15:19:54.179813 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.179793    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"]
Jul 07 15:19:54.184162 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.184118    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-8p5bm"]
Jul 07 15:19:54.796486 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:54.792693    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d731d089-94f4-4860-a873-f9bcd2c97169" path="/var/lib/kubelet/pods/d731d089-94f4-4860-a873-f9bcd2c97169/volumes"
Jul 07 15:19:57.681240 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:57.681211    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:19:58.190506 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:19:58.190474    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:21:37.272467 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:21:37.272429    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:22:08.845742 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:22:08.845648    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:22:12.950878 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:22:12.950838    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:22:13.058392 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:22:13.058356    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:24:13.232802 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:24:13.232764    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:24:18.280681 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:24:18.280640    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:24:18.434482 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:24:18.434444    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:24:48.459207 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:24:48.459163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:24:54.629957 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:24:54.629919    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:24:54.728619 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:24:54.728581    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:25:24.943455 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:24.943421    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:25:25.091979 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:25.091940    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:25:25.188362 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:25.188331    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:25:28.122990 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:28.122952    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-266nr_bde44302-557d-46b5-9d01-c43bcc78d2ec/global-pull-secret-syncer/0.log"
Jul 07 15:25:28.301275 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:28.301238    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log"
Jul 07 15:25:28.485046 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:28.485018    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-158.ec2.internal_7649a49abd2efd166fe10d91dbf1337c/haproxy/0.log"
Jul 07 15:25:31.089835 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:31.089805    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:25:31.199928 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:31.199896    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfssq_3b8a0f26-98e2-4c8f-91db-4b6a4c6bf44c/konnectivity-agent/0.log" limit=1024
Jul 07 15:25:33.721164 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:33.721118    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-nlqbt_fd1290ae-d56e-4d69-b315-e4a4346432a4/monitoring-plugin/0.log"
Jul 07 15:25:33.786335 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:33.786307    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9fdsp_6991c135-1f17-45fd-bdcd-7c94f4365586/node-exporter/0.log"
Jul 07 15:25:33.843382 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:33.843353    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9fdsp_6991c135-1f17-45fd-bdcd-7c94f4365586/kube-rbac-proxy/0.log"
Jul 07 15:25:33.880198 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:33.880169    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9fdsp_6991c135-1f17-45fd-bdcd-7c94f4365586/init-textfile/0.log"
Jul 07 15:25:37.035363 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.035331    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"]
Jul 07 15:25:37.035755 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.035635    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d731d089-94f4-4860-a873-f9bcd2c97169" containerName="kube-rbac-proxy"
Jul 07 15:25:37.035755 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.035648    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d731d089-94f4-4860-a873-f9bcd2c97169" containerName="kube-rbac-proxy"
Jul 07 15:25:37.035755 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.035661    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d731d089-94f4-4860-a873-f9bcd2c97169" containerName="kserve-container"
Jul 07 15:25:37.035755 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.035669    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d731d089-94f4-4860-a873-f9bcd2c97169" containerName="kserve-container"
Jul 07 15:25:37.035755 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.035681    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d731d089-94f4-4860-a873-f9bcd2c97169" containerName="storage-initializer"
Jul 07 15:25:37.035755 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.035688    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d731d089-94f4-4860-a873-f9bcd2c97169" containerName="storage-initializer"
Jul 07 15:25:37.035940 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.035760    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d731d089-94f4-4860-a873-f9bcd2c97169" containerName="kserve-container"
Jul 07 15:25:37.035940 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.035769    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d731d089-94f4-4860-a873-f9bcd2c97169" containerName="kube-rbac-proxy"
Jul 07 15:25:37.038724 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.038704    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.041795 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.041776    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2kxcl\"/\"kube-root-ca.crt\""
Jul 07 15:25:37.041915 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.041793    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2kxcl\"/\"openshift-service-ca.crt\""
Jul 07 15:25:37.042215 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.042199    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2kxcl\"/\"default-dockercfg-kwhgw\""
Jul 07 15:25:37.050221 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.050202    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"]
Jul 07 15:25:37.142835 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.142802    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scftm\" (UniqueName: \"kubernetes.io/projected/2708685a-2790-4245-b777-6d27ca72f193-kube-api-access-scftm\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.143006 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.142841    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2708685a-2790-4245-b777-6d27ca72f193-proc\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.143006 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.142866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2708685a-2790-4245-b777-6d27ca72f193-sys\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.143006 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.142954    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2708685a-2790-4245-b777-6d27ca72f193-lib-modules\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.143006 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.142983    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2708685a-2790-4245-b777-6d27ca72f193-podres\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.244189 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.244149    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2708685a-2790-4245-b777-6d27ca72f193-sys\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.244322 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.244204    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2708685a-2790-4245-b777-6d27ca72f193-lib-modules\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.244322 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.244235    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2708685a-2790-4245-b777-6d27ca72f193-podres\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.244322 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.244280    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2708685a-2790-4245-b777-6d27ca72f193-sys\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.244322 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.244287    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scftm\" (UniqueName: \"kubernetes.io/projected/2708685a-2790-4245-b777-6d27ca72f193-kube-api-access-scftm\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.244519 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.244363    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2708685a-2790-4245-b777-6d27ca72f193-proc\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.244519 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.244388    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2708685a-2790-4245-b777-6d27ca72f193-podres\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.244519 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.244389    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2708685a-2790-4245-b777-6d27ca72f193-lib-modules\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.244519 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.244436    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2708685a-2790-4245-b777-6d27ca72f193-proc\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.253206 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.253182    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scftm\" (UniqueName: \"kubernetes.io/projected/2708685a-2790-4245-b777-6d27ca72f193-kube-api-access-scftm\") pod \"perf-node-gather-daemonset-mqqnc\" (UID: \"2708685a-2790-4245-b777-6d27ca72f193\") " pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.340179 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.340079    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5788799c7-592fk_353c01c7-cb71-4c54-bedd-8cec691bc245/console/0.log"
Jul 07 15:25:37.348088 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.348067    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:37.473533 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:37.473510    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"]
Jul 07 15:25:37.476255 ip-10-0-140-158 kubenswrapper[2582]: W0707 15:25:37.476230    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2708685a_2790_4245_b777_6d27ca72f193.slice/crio-c2c10e0737a97c43cb668a620fc29c06c2e33d6479bb62d6052d6dc6a2e580e8 WatchSource:0}: Error finding container c2c10e0737a97c43cb668a620fc29c06c2e33d6479bb62d6052d6dc6a2e580e8: Status 404 returned error can't find the container with id c2c10e0737a97c43cb668a620fc29c06c2e33d6479bb62d6052d6dc6a2e580e8
Jul 07 15:25:38.388754 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:38.388721    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc" event={"ID":"2708685a-2790-4245-b777-6d27ca72f193","Type":"ContainerStarted","Data":"793878bc757ee744f44846341f757347e8ada2c5e3fa48e2f2ddd194b96b73a5"}
Jul 07 15:25:38.389117 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:38.388771    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc" event={"ID":"2708685a-2790-4245-b777-6d27ca72f193","Type":"ContainerStarted","Data":"c2c10e0737a97c43cb668a620fc29c06c2e33d6479bb62d6052d6dc6a2e580e8"}
Jul 07 15:25:38.389117 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:38.388860    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:38.413413 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:38.413367    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc" podStartSLOduration=1.413351662 podStartE2EDuration="1.413351662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:25:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:25:38.411193435 +0000 UTC m=+1742.084799490" watchObservedRunningTime="2026-07-07 15:25:38.413351662 +0000 UTC m=+1742.086957717"
Jul 07 15:25:39.191467 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:39.191438    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nvbd4_54adbea3-fb47-41d6-8e45-1388111db0bb/dns/0.log"
Jul 07 15:25:39.232564 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:39.232528    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nvbd4_54adbea3-fb47-41d6-8e45-1388111db0bb/kube-rbac-proxy/0.log"
Jul 07 15:25:39.395326 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:39.395297    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x9g98_caee8dda-918b-405f-af0c-f7bb812ab885/dns-node-resolver/0.log"
Jul 07 15:25:39.995698 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:39.995656    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gnn6l_c132d28e-79ff-4bef-8d61-b5a24f10a279/node-ca/0.log"
Jul 07 15:25:41.357003 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:41.356968    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rwg52_797b0a24-ae0e-4ff0-abd2-f6654de72287/serve-healthcheck-canary/0.log"
Jul 07 15:25:41.954275 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:41.954243    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t24sv_1cf8df9c-54d8-426d-9a67-4edfb2441335/kube-rbac-proxy/0.log"
Jul 07 15:25:41.976012 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:41.975984    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t24sv_1cf8df9c-54d8-426d-9a67-4edfb2441335/exporter/0.log"
Jul 07 15:25:42.001481 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:42.001454    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t24sv_1cf8df9c-54d8-426d-9a67-4edfb2441335/extractor/0.log"
Jul 07 15:25:44.229986 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:44.229949    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-666b75c8f7-llgnq_f787152a-b959-49dd-b3c8-9627ab5a4b67/manager/0.log"
Jul 07 15:25:44.403543 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:44.403513    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2kxcl/perf-node-gather-daemonset-mqqnc"
Jul 07 15:25:44.460265 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:44.460237    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-gp5j5_ba5555f8-b28e-44e2-877f-96c18c8dcbb0/manager/0.log"
Jul 07 15:25:44.657321 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:44.657237    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-r9nlx_84a1b661-74df-490e-91a7-90ed02cbefe1/seaweedfs/0.log"
Jul 07 15:25:44.684762 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:44.684736    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-tls-custom-8b44d767d-c5xc5_c945218a-1cd3-47d1-aceb-4bf07519db2f/seaweedfs-tls-custom/0.log"
Jul 07 15:25:49.630735 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:49.630690    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-xc6b9_956cfaa6-0356-40c3-a659-a7716aac6580/migrator/0.log"
Jul 07 15:25:49.662176 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:49.662146    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-xc6b9_956cfaa6-0356-40c3-a659-a7716aac6580/graceful-termination/0.log"
Jul 07 15:25:51.020432 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:51.020401    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9k67d_c23ce787-ca1c-4ee5-84b7-2b7aae623f51/kube-multus/0.log"
Jul 07 15:25:51.411406 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:51.411336    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xdxzl_8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a/kube-multus-additional-cni-plugins/0.log"
Jul 07 15:25:51.433478 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:51.433449    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xdxzl_8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a/egress-router-binary-copy/0.log"
Jul 07 15:25:51.458185 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:51.458156    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xdxzl_8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a/cni-plugins/0.log"
Jul 07 15:25:51.480597 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:51.480572    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xdxzl_8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a/bond-cni-plugin/0.log"
Jul 07 15:25:51.501801 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:51.501776    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xdxzl_8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a/routeoverride-cni/0.log"
Jul 07 15:25:51.523765 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:51.523739    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xdxzl_8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a/whereabouts-cni-bincopy/0.log"
Jul 07 15:25:51.549009 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:51.548988    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xdxzl_8aecdfb7-b67c-4208-a8ce-6bd6e1ee2d0a/whereabouts-cni/0.log"
Jul 07 15:25:51.607647 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:51.607624    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dzg7v_b0469d85-dc53-4bb1-891e-99ed1d55fee8/network-metrics-daemon/0.log"
Jul 07 15:25:51.629201 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:51.629174    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dzg7v_b0469d85-dc53-4bb1-891e-99ed1d55fee8/kube-rbac-proxy/0.log"
Jul 07 15:25:52.995927 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:52.995901    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9bhb_54a7480b-096b-4bdf-8352-632d0216e013/ovn-controller/0.log"
Jul 07 15:25:53.024018 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:53.023993    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9bhb_54a7480b-096b-4bdf-8352-632d0216e013/ovn-acl-logging/0.log"
Jul 07 15:25:53.053897 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:53.053870    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9bhb_54a7480b-096b-4bdf-8352-632d0216e013/kube-rbac-proxy-node/0.log"
Jul 07 15:25:53.078487 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:53.078466    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9bhb_54a7480b-096b-4bdf-8352-632d0216e013/kube-rbac-proxy-ovn-metrics/0.log"
Jul 07 15:25:53.099719 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:53.099691    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9bhb_54a7480b-096b-4bdf-8352-632d0216e013/northd/0.log"
Jul 07 15:25:53.125700 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:53.125674    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9bhb_54a7480b-096b-4bdf-8352-632d0216e013/nbdb/0.log"
Jul 07 15:25:53.152671 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:53.152615    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9bhb_54a7480b-096b-4bdf-8352-632d0216e013/sbdb/0.log"
Jul 07 15:25:53.215558 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:53.215518    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9bhb_54a7480b-096b-4bdf-8352-632d0216e013/ovnkube-controller/0.log"
Jul 07 15:25:54.346306 ip-10-0-140-158 kubenswrapper[2582]: I0707 15:25:54.346271    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-7dqwt_b8bd0203-d82d-4727-b612-ef4669b4eab3/network-check-target-container/0.log"