Jul 07 14:56:43.868527 ip-10-0-130-158 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 14:56:44.284169 ip-10-0-130-158 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:56:44.284169 ip-10-0-130-158 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 14:56:44.284169 ip-10-0-130-158 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:56:44.284169 ip-10-0-130-158 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 14:56:44.284169 ip-10-0-130-158 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:56:44.284867 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.284228    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 14:56:44.288081 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288063    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:44.288081 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288079    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:44.288081 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288083    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288087    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288091    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288095    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288099    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288102    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288105    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288108    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288111    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288114    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288117    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288119    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288129    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288134    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288137    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288140    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288143    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288147    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288150    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288152    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:44.288169 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288155    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288158    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288161    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288163    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288166    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288169    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288172    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288175    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288178    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288181    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288184    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288188    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288191    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288194    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288197    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288200    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288202    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288205    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288209    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288213    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:44.288726 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288217    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288220    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288222    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288225    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288228    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288231    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288233    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288236    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288239    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288241    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288244    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288247    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288251    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288254    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288258    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288262    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288265    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288268    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288271    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:44.289241 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288275    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288278    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288281    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288284    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288287    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288290    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288292    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288295    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288299    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288302    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288305    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288308    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288311    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288314    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288316    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288319    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288322    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288339    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288343    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288348    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:44.289750 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288352    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288356    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288359    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288363    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288365    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288369    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288372    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288375    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288378    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288381    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288384    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288387    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288390    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288393    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288395    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288398    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288401    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.288404    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289156    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289163    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:44.290276 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289166    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289169    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289173    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289177    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289180    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289183    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289186    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289189    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289192    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289195    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289198    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289201    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289204    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289207    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289209    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289213    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289216    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289219    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289222    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289225    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:44.290815 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289228    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289233    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289237    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289240    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289244    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289247    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289250    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289253    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289256    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289259    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289262    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289265    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289268    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289270    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289273    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289276    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289280    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289282    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289285    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:44.291403 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289288    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289290    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289293    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289296    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289299    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289301    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289304    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289307    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289311    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289313    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289316    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289319    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289322    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289342    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289345    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289348    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289352    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289355    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289358    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289361    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:44.291892 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289363    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289366    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289369    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289372    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289375    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289380    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289383    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289386    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289389    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289392    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289396    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289399    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289401    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289404    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289407    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289410    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289413    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289415    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289419    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:44.292445 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289423    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289428    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289431    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289434    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289438    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289440    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289443    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289446    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289449    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289452    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289455    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289458    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289461    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289464    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289467    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289469    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289472    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289475    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289478    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289881    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:44.292931 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289885    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289888    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289891    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289895    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289897    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289900    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289903    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289906    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289909    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289912    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289915    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289917    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289920    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289923    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289927    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289932    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289936    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289939    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289942    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:44.293457 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289945    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289948    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289951    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289953    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289956    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289959    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289962    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289966    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289969    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289972    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289975    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289978    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289981    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289983    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289986    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289989    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289992    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289995    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.289999    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290002    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:44.293955 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290005    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290008    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290011    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290013    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290016    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290019    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290022    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290024    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290027    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290031    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290033    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290036    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290039    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290042    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290045    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290047    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290050    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290053    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290056    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290060    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:44.294482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290062    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290065    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290068    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290070    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290073    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290075    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290078    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290081    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290084    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290087    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290091    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290093    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290096    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290099    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290102    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290105    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290107    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290110    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290113    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:44.294994 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290116    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290119    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290121    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290124    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290127    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290129    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290132    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290135    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290137    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290140    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290142    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290145    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290148    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290151    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290155    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290158    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290161    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290164    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290167    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290170    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:44.295503 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290248    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290260    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290266    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290274    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290279    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290283    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290288    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290293    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290297    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290300    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290304    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290308    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290311    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290315    2569 flags.go:64] FLAG: --cgroup-root=""
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290319    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290322    2569 flags.go:64] FLAG: --client-ca-file=""
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290345    2569 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290348    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290353    2569 flags.go:64] FLAG: --cluster-domain=""
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290356    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290360    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290363    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290367    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290371    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 14:56:44.296016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290375    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290378    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290382    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290385    2569 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290388    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290392    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290395    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290399    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290403    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290407    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290410    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290414    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290418    2569 flags.go:64] FLAG: --enable-server="true"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290422    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290426    2569 flags.go:64] FLAG: --event-burst="100"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290429    2569 flags.go:64] FLAG: --event-qps="50"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290433    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290436    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290439    2569 flags.go:64] FLAG: --eviction-hard=""
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290443    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290447    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290450    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290453    2569 flags.go:64] FLAG: --eviction-soft=""
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290456    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290459    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 14:56:44.296691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290463    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290466    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290470    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290473    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290476    2569 flags.go:64] FLAG: --feature-gates=""
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290480    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290483    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290487    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290491    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290494    2569 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290497    2569 flags.go:64] FLAG: --help="false"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290501    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290504    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290507    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290511    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290514    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290518    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290523    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290529    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290532    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290536    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290539    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290544    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290547    2569 flags.go:64] FLAG: --kube-reserved=""
Jul 07 14:56:44.297375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290550    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290553    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290557    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290560    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290563    2569 flags.go:64] FLAG: --lock-file=""
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290566    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290569    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290573    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290579    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290582    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290585    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290588    2569 flags.go:64] FLAG: --logging-format="text"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290591    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290595    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290598    2569 flags.go:64] FLAG: --manifest-url=""
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290601    2569 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290606    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290609    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290614    2569 flags.go:64] FLAG: --max-pods="110"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290617    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290620    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290623    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290627    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290630    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290633    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 14:56:44.298024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290638    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290646    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290650    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290654    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290657    2569 flags.go:64] FLAG: --pod-cidr=""
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290661    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290667    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290670    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290674    2569 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290677    2569 flags.go:64] FLAG: --port="10250"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290680    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290683    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f6ae521c2669d8ad"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290687    2569 flags.go:64] FLAG: --qos-reserved=""
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290690    2569 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290693    2569 flags.go:64] FLAG: --register-node="true"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290696    2569 flags.go:64] FLAG: --register-with-taints=""
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290701    2569 flags.go:64] FLAG: --registry-burst="10"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290704    2569 flags.go:64] FLAG: --registry-qps="5"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290707    2569 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290710    2569 flags.go:64] FLAG: --reserved-memory=""
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290715    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290718    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290722    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290725    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290728    2569 flags.go:64] FLAG: --runonce="false"
Jul 07 14:56:44.298685 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290731    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290734    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290738    2569 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290741    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290744    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290747    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290750    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290755    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290758    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290764    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290767    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290770    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290773    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290777    2569 flags.go:64] FLAG: --system-cgroups=""
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290780    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290786    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290789    2569 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290792    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290797    2569 flags.go:64] FLAG: --tls-min-version=""
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290800    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290803    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290806    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290809    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290813    2569 flags.go:64] FLAG: --v="2"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290817    2569 flags.go:64] FLAG: --version="false"
Jul 07 14:56:44.299344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290821    2569 flags.go:64] FLAG: --vmodule=""
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290828    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.290832    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290978    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290983    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290986    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290990    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290993    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290996    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.290999    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291001    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291004    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291007    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291010    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291014    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291017    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291022    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291025    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291027    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291030    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:44.300047 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291033    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291036    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291039    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291042    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291045    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291048    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291051    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291053    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291056    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291059    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291061    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291064    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291067    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291070    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291073    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291076    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291078    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291081    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291086    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:44.300628 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291090    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291093    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291096    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291099    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291102    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291105    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291107    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291111    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291114    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291119    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291122    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291124    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291127    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291130    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291133    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291136    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291139    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291142    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291145    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291147    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:44.301131 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291150    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291153    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291156    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291158    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291161    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291164    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291167    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291170    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291173    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291176    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291178    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291181    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291184    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291186    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291189    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291192    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291194    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291197    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291200    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291204    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:44.301674 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291206    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291211    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291214    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291216    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291219    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291222    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291225    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291228    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291231    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291234    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291238    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291242    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291245    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291248    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291251    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291254    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291257    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291259    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291262    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:44.302200 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291265    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291268    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291272    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.291275    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.292239    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.299250    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.299270    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.299288    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.299294    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299376    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299381    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299385    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299389    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299392    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:44.302721 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299395    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299398    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299401    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299404    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299407    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299410    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299413    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299415    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299418    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299421    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299424    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299427    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299429    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299432    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299435    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299438    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299441    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299443    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299446    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299449    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:44.303123 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299452    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299455    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299457    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299460    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299465    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299468    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299471    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299474    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299476    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299479    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299483    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299486    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299489    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299491    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299494    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299498    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299501    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299504    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299506    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:44.303712 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299509    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299528    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299532    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299536    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299539    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299543    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299546    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299549    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299552    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299555    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299558    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299561    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299564    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299566    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299569    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299572    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299575    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299579    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299584    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:44.304212 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299588    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299592    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299596    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299600    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299603    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299606    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299609    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299614    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299618    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299621    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299625    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299628    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299631    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299634    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299637    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299640    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299643    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299646    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299650    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299652    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:44.304753 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299655    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299658    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299661    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299664    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299667    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299670    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299673    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299676    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299679    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299683    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299685    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299688    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299692    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299695    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299697    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:44.305269 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299701    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.299706    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299827    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299832    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299835    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299838    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299841    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299844    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299847    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299850    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299852    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299855    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299858    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299861    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299863    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:56:44.305677 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299866    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299869    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299872    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299874    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299878    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299881    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299883    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299886    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299889    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299892    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299894    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299898    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299901    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299903    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299906    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299910    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299912    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299915    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299919    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:56:44.306082 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299923    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299926    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299929    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299932    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299934    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299938    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299940    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299943    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299946    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299949    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299952    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299954    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299959    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299962    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299966    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299968    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299971    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299974    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299976    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:56:44.306588 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299979    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299982    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299985    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299987    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299990    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299993    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299995    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.299998    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300001    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300004    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300007    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300009    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300012    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300015    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300018    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300020    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300023    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300026    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300028    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300031    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:56:44.307095 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300034    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300036    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300039    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300042    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300044    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300047    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300050    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300053    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300056    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300059    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300061    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300064    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300067    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300070    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300073    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300075    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300078    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300081    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300084    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300087    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:56:44.307779 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300089    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300092    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300095    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300098    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300101    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300105    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300108    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:44.300111    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.300116    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.300220    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.302625    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.304149    2569 server.go:1019] "Starting client certificate rotation"
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.304270    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 14:56:44.308286 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.304309    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 14:56:44.308639 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.306857    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 14:56:44.311391 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.311361    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 14:56:44.321051 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.321030    2569 log.go:25] "Validated CRI v1 runtime API"
Jul 07 14:56:44.324431 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.324408    2569 log.go:25] "Validated CRI v1 image API"
Jul 07 14:56:44.326273 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.326253    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 14:56:44.328779 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.328754    2569 fs.go:135] Filesystem UUIDs: map[68102a72-041c-4e4c-84a0-2621a420d911:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fe1cd854-8529-4803-8ed6-c19363889ff0:/dev/nvme0n1p4]
Jul 07 14:56:44.328836 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.328779    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 14:56:44.334338 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.334199    2569 manager.go:216] Machine: {Timestamp:2026-07-07 14:56:44.332858286 +0000 UTC m=+0.345254848 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3129988 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2912a91a7260b19d5db581cfdd959a SystemUUID:ec2912a9-1a72-60b1-9d5d-b581cfdd959a BootID:d1d5257c-6b52-4561-a398-197e5fef3fe5 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f1:6e:41:89:13 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f1:6e:41:89:13 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:9f:61:60:ca:ce Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 14:56:44.334338 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.334315    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 14:56:44.334534 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.334418    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 14:56:44.338224 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.338190    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 14:56:44.338371 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.338354    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 14:56:44.338421 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.338227    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-158.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 14:56:44.339566 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.339555    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 14:56:44.339602 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.339570    2569 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 14:56:44.339602 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.339578    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 14:56:44.340295 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.340283    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 14:56:44.341719 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.341707    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 14:56:44.345132 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.345119    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 14:56:44.345251 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.345241    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 14:56:44.345292 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.345274    2569 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 14:56:44.345292 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.345286    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 14:56:44.345373 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.345300    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 14:56:44.345373 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.345313    2569 kubelet.go:398] "Adding apiserver pod source"
Jul 07 14:56:44.345373 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.345323    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 14:56:44.345796 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.345781    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 14:56:44.345835 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.345805    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 14:56:44.348731 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.348710    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 14:56:44.350650 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.350636    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 14:56:44.350699 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.350669    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 14:56:44.351800 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.351787    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 14:56:44.351854 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.351805    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 14:56:44.351854 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.351813    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 14:56:44.351854 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.351819    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 14:56:44.351854 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.351829    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 14:56:44.351854 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.351835    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 14:56:44.351854 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.351844    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 14:56:44.351854 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.351849    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 14:56:44.351854 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.351857    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 14:56:44.352065 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.351862    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 14:56:44.352065 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.351872    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 14:56:44.352065 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.351877    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 14:56:44.352752 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.352742    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 14:56:44.352752 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.352752    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 14:56:44.356377 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.356304    2569 server.go:1268] "Started kubelet"
Jul 07 14:56:44.360585 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.356403    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 14:56:44.360709 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.360662    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 14:56:44.360920 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.356399    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 14:56:44.361300 ip-10-0-130-158 systemd[1]: Started Kubernetes Kubelet.
Jul 07 14:56:44.363051 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.362547    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 14:56:44.363296 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.363132    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 14:56:44.363449 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.363302    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-158.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 14:56:44.363449 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.363355    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-158.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 14:56:44.363710 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.363688    2569 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 14:56:44.367015 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.366992    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6htpk"
Jul 07 14:56:44.368750 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.367702    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-158.ec2.internal.18c009c7603a0c51  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-158.ec2.internal,UID:ip-10-0-130-158.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-158.ec2.internal,},FirstTimestamp:2026-07-07 14:56:44.356275281 +0000 UTC m=+0.368671821,LastTimestamp:2026-07-07 14:56:44.356275281 +0000 UTC m=+0.368671821,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-158.ec2.internal,}"
Jul 07 14:56:44.369290 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.369269    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 14:56:44.370784 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.370763    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 14:56:44.370871 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.370860    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:44.370925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.370771    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 14:56:44.371373 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.371334    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 14:56:44.371373 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.371337    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 14:56:44.371373 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.371366    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 14:56:44.371577 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.371481    2569 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 14:56:44.371577 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.371516    2569 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 14:56:44.371929 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.371896    2569 factory.go:153] Registering CRI-O factory
Jul 07 14:56:44.371929 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.371917    2569 factory.go:223] Registration of the crio container factory successfully
Jul 07 14:56:44.372069 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.371976    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 14:56:44.372069 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.371986    2569 factory.go:55] Registering systemd factory
Jul 07 14:56:44.372069 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.371995    2569 factory.go:223] Registration of the systemd container factory successfully
Jul 07 14:56:44.372069 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.372019    2569 factory.go:103] Registering Raw factory
Jul 07 14:56:44.372069 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.372033    2569 manager.go:1196] Started watching for new ooms in manager
Jul 07 14:56:44.372647 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.372632    2569 manager.go:349] Starting recovery of all containers
Jul 07 14:56:44.378454 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.378424    2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-158.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 14:56:44.378589 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.378563    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 14:56:44.381351 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.381310    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6htpk"
Jul 07 14:56:44.384217 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.384099    2569 manager.go:354] Recovery completed
Jul 07 14:56:44.385946 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.385922    2569 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 07 14:56:44.389015 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.389003    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:44.391022 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.391007    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:44.391089 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.391037    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:44.391089 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.391050    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:44.391618 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.391602    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 14:56:44.391618 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.391612    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:44.391748 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.391613    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 14:56:44.391748 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.391675    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 14:56:44.392403 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.392387    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:44.392460 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.392421    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:44.392460 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.392435    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:44.392868 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.392854    2569 policy_none.go:49] "None policy: Start"
Jul 07 14:56:44.392907 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.392872    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 14:56:44.392907 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.392882    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 14:56:44.393361 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.393276    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-158.ec2.internal.18c009c7624c3ac7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-158.ec2.internal,UID:ip-10-0-130-158.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-158.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-158.ec2.internal,},FirstTimestamp:2026-07-07 14:56:44.391021255 +0000 UTC m=+0.403417794,LastTimestamp:2026-07-07 14:56:44.391021255 +0000 UTC m=+0.403417794,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-158.ec2.internal,}"
Jul 07 14:56:44.394009 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.393997    2569 policy_none.go:47] "Start"
Jul 07 14:56:44.436103 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.436072    2569 manager.go:337] "Starting Device Plugin manager"
Jul 07 14:56:44.436229 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.436117    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 14:56:44.436229 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.436129    2569 server.go:87] "Starting device plugin registration server"
Jul 07 14:56:44.436515 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.436497    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 14:56:44.436600 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.436516    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 14:56:44.436600 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.436582    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 14:56:44.436708 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.436663    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 14:56:44.436708 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.436672    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 14:56:44.437332 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.437307    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 14:56:44.437442 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.437362    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:44.509724 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.509655    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 14:56:44.510919 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.510902    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 14:56:44.510978 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.510943    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 14:56:44.510978 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.510971    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 14:56:44.511064 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.511009    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 14:56:44.513587 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.513557    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:44.536897 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.536875    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:44.538203 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.538177    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:44.538203 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.538206    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:44.538383 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.538221    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:44.538383 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.538250    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.547858 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.547840    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.547918 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.547864    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-158.ec2.internal\": node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:44.573186 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.573158    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:44.611300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.611271    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-158.ec2.internal"]
Jul 07 14:56:44.611431 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.611339    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:44.612927 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.612912    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:44.612997 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.612942    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:44.612997 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.612953    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:44.614156 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.614143    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:44.614254 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.614237    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.614302 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.614287    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:44.614920 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.614897    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:44.615014 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.614924    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:44.615014 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.614938    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:44.615014 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.614903    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:44.615014 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.614996    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:44.615014 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.615007    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:44.616114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.616098    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.616196 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.616129    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:56:44.617088 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.617063    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:56:44.617161 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.617095    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:56:44.617161 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.617112    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:56:44.640060 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.640036    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-158.ec2.internal\" not found" node="ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.644624 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.644608    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-158.ec2.internal\" not found" node="ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.673266 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.673219    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:44.673399 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.673304    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/556c0bc2b69d90a68099396d7b905ebb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal\" (UID: \"556c0bc2b69d90a68099396d7b905ebb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.673399 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.673329    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/556c0bc2b69d90a68099396d7b905ebb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal\" (UID: \"556c0bc2b69d90a68099396d7b905ebb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.673399 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.673347    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/831e9ec52a3238957c7f1fb3d14b894f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-158.ec2.internal\" (UID: \"831e9ec52a3238957c7f1fb3d14b894f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.773599 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.773500    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:44.773599 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.773524    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/556c0bc2b69d90a68099396d7b905ebb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal\" (UID: \"556c0bc2b69d90a68099396d7b905ebb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.773599 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.773561    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/556c0bc2b69d90a68099396d7b905ebb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal\" (UID: \"556c0bc2b69d90a68099396d7b905ebb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.773599 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.773583    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/831e9ec52a3238957c7f1fb3d14b894f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-158.ec2.internal\" (UID: \"831e9ec52a3238957c7f1fb3d14b894f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.773837 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.773651    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/831e9ec52a3238957c7f1fb3d14b894f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-158.ec2.internal\" (UID: \"831e9ec52a3238957c7f1fb3d14b894f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.773837 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.773651    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/556c0bc2b69d90a68099396d7b905ebb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal\" (UID: \"556c0bc2b69d90a68099396d7b905ebb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.773837 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.773718    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/556c0bc2b69d90a68099396d7b905ebb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal\" (UID: \"556c0bc2b69d90a68099396d7b905ebb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.873644 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.873591    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:44.946999 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.946954    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.948144 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:44.948130    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:44.974704 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:44.974665    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:45.076724 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:45.076612    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:45.178701 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:45.178664    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:45.279420 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:45.279386    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:45.303864 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:45.303824    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 14:56:45.304425 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:45.304045    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 14:56:45.380588 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:45.380510    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:45.383555 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:45.383533    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 14:51:45 +0000 UTC" deadline="2028-02-12 18:05:52.983168082 +0000 UTC"
Jul 07 14:56:45.383610 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:45.383556    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14043h9m7.599615566s"
Jul 07 14:56:45.481419 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:45.481386    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-158.ec2.internal\" not found"
Jul 07 14:56:45.515923 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:45.515898    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:45.533080 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:45.533050    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod556c0bc2b69d90a68099396d7b905ebb.slice/crio-28ae783fc7f1e1bdff249616df1a5c2334103a95c9f8befab0090e4dedab452f WatchSource:0}: Error finding container 28ae783fc7f1e1bdff249616df1a5c2334103a95c9f8befab0090e4dedab452f: Status 404 returned error can't find the container with id 28ae783fc7f1e1bdff249616df1a5c2334103a95c9f8befab0090e4dedab452f
Jul 07 14:56:45.539619 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:45.539598    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 14:56:45.558156 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:45.558132    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod831e9ec52a3238957c7f1fb3d14b894f.slice/crio-bdf927591819765c5f2bf3d9e4b91ddea4d46ad5d609535b35816f5458aef9c5 WatchSource:0}: Error finding container bdf927591819765c5f2bf3d9e4b91ddea4d46ad5d609535b35816f5458aef9c5: Status 404 returned error can't find the container with id bdf927591819765c5f2bf3d9e4b91ddea4d46ad5d609535b35816f5458aef9c5
Jul 07 14:56:45.572045 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:45.572020    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:45.580859 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:45.580843    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 14:56:45.582915 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:45.582898    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal"
Jul 07 14:56:45.593424 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:45.593408    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:45.597676 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:45.597654    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:45.598783 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:45.598768    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 14:56:46.324133 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.324093    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:56:46.346961 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.346924    2569 apiserver.go:52] "Watching apiserver"
Jul 07 14:56:46.359289 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.359218    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 14:56:46.363220 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.363184    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-gpxk2","kube-system/konnectivity-agent-9zk5l","kube-system/kube-apiserver-proxy-ip-10-0-130-158.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k","openshift-cluster-node-tuning-operator/tuned-5c8ng","openshift-dns/node-resolver-cmwns","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal","openshift-multus/multus-2xxrr","openshift-multus/multus-additional-cni-plugins-2nvp9","kube-system/global-pull-secret-syncer-4rqtz","openshift-image-registry/node-ca-ftfzh","openshift-multus/network-metrics-daemon-s5zzk","openshift-network-diagnostics/network-check-target-5w2jq","openshift-network-operator/iptables-alerter-8dg7c"]
Jul 07 14:56:46.364924 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.364896    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:46.365054 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.364993    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:56:46.366297 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.366275    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:46.366432 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.366348    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:56:46.367520 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.367467    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8dg7c"
Jul 07 14:56:46.369269 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.369227    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.370374 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.370353    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:56:46.370834 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.370552    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 14:56:46.370834 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.370624    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 14:56:46.370834 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.370695    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z2mgc\""
Jul 07 14:56:46.371124 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.371110    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 14:56:46.371279 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.371241    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.371990 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.371972    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 14:56:46.372814 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.372798    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cmwns"
Jul 07 14:56:46.373121 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.373100    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v9s2t\""
Jul 07 14:56:46.373482 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.373459    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 14:56:46.373482 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.373482    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 14:56:46.374616 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.374552    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:56:46.374616 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.374580    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.375068 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.374564    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 14:56:46.375778 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.375531    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.376840 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.376818    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 14:56:46.377799 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.377777    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9zk5l"
Jul 07 14:56:46.379302 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.378457    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 14:56:46.379302 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.378492    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-w9sqg\""
Jul 07 14:56:46.380609 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.380587    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 14:56:46.380852 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.380836    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 14:56:46.381323 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.381307    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zxnk7\""
Jul 07 14:56:46.381522 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.381494    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 14:56:46.381588 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.381565    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zf26w\""
Jul 07 14:56:46.381698 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.381679    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 14:56:46.382259 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.382239    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6wj7v\""
Jul 07 14:56:46.382394 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.382374    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 14:56:46.382453 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.382404    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.382789 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.382768    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5dwqx\""
Jul 07 14:56:46.382866 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.382811    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 14:56:46.384424 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.383281    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 14:56:46.384424 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.383556    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ftfzh"
Jul 07 14:56:46.384424 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.383824    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:46.384424 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.383905    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:56:46.384906 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.384884    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 14:56:46.387680 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387659    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 14:56:46.387793 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387655    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3418602d-7767-4f00-b89f-8903c6faf891-tmp\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.387793 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387714    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-host\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.387793 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387721    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bml5m\""
Jul 07 14:56:46.387793 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387742    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwlsl\" (UniqueName: \"kubernetes.io/projected/cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093-kube-api-access-qwlsl\") pod \"iptables-alerter-8dg7c\" (UID: \"cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093\") " pod="openshift-network-operator/iptables-alerter-8dg7c"
Jul 07 14:56:46.387793 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387769    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.388094 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387825    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-device-dir\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.388094 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387853    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-etc-selinux\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.388094 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387878    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-sys-fs\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.388094 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387905    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 14:56:46.388094 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387659    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 14:56:46.388094 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387906    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 14:56:46.388094 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.387905    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093-host-slash\") pod \"iptables-alerter-8dg7c\" (UID: \"cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093\") " pod="openshift-network-operator/iptables-alerter-8dg7c"
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388100    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388112    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-modprobe-d\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388152    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-sysconfig\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388177    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-systemd\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388207    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-var-lib-kubelet\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388231    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3418602d-7767-4f00-b89f-8903c6faf891-etc-tuned\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388258    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093-iptables-alerter-script\") pod \"iptables-alerter-8dg7c\" (UID: \"cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093\") " pod="openshift-network-operator/iptables-alerter-8dg7c"
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388281    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388293    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-sysctl-d\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388317    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-run\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388363    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9wtp\" (UniqueName: \"kubernetes.io/projected/b2a0a171-7c99-4f1b-b735-8f0083a4094b-kube-api-access-d9wtp\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388389    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03e07f30-8257-4a4b-bb01-042bb672a64c-tmp-dir\") pod \"node-resolver-cmwns\" (UID: \"03e07f30-8257-4a4b-bb01-042bb672a64c\") " pod="openshift-dns/node-resolver-cmwns"
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388418    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f99558a-08a5-4532-a131-eb30f2278bb5-kubelet-config\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:46.388464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388449    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-lib-modules\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.389083 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388482    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-kubernetes\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.389083 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388504    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-sys\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.389083 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388531    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-socket-dir\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.389083 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388574    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-registration-dir\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.389083 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388649    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-sysctl-conf\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.389083 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdmrs\" (UniqueName: \"kubernetes.io/projected/3418602d-7767-4f00-b89f-8903c6faf891-kube-api-access-hdmrs\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.389083 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388744    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktt92\" (UniqueName: \"kubernetes.io/projected/03e07f30-8257-4a4b-bb01-042bb672a64c-kube-api-access-ktt92\") pod \"node-resolver-cmwns\" (UID: \"03e07f30-8257-4a4b-bb01-042bb672a64c\") " pod="openshift-dns/node-resolver-cmwns"
Jul 07 14:56:46.389083 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388788    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wwxg\" (UniqueName: \"kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg\") pod \"network-check-target-5w2jq\" (UID: \"213be297-4d57-4440-9531-e4121fe87b19\") " pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:46.389083 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388813    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03e07f30-8257-4a4b-bb01-042bb672a64c-hosts-file\") pod \"node-resolver-cmwns\" (UID: \"03e07f30-8257-4a4b-bb01-042bb672a64c\") " pod="openshift-dns/node-resolver-cmwns"
Jul 07 14:56:46.389083 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388836    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f99558a-08a5-4532-a131-eb30f2278bb5-dbus\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:46.389083 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.388877    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:46.393132 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.393045    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 14:56:46.393888 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.393227    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jzhpl\""
Jul 07 14:56:46.393888 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.393329    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 14:56:46.393888 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.393375    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 14:56:46.397003 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.396968    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 14:56:46.423184 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.423150    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8nnq9"
Jul 07 14:56:46.432277 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.432249    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8nnq9"
Jul 07 14:56:46.472687 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.472660    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 14:56:46.489491 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489459    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwlsl\" (UniqueName: \"kubernetes.io/projected/cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093-kube-api-access-qwlsl\") pod \"iptables-alerter-8dg7c\" (UID: \"cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093\") " pod="openshift-network-operator/iptables-alerter-8dg7c"
Jul 07 14:56:46.489624 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489502    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-device-dir\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.489624 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489535    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b78593d3-7844-497b-96a9-17cd45cb49aa-system-cni-dir\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.489743 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489630    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-device-dir\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.489803 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489737    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-sysconfig\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.489803 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489795    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-var-lib-kubelet\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.489908 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489825    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b78593d3-7844-497b-96a9-17cd45cb49aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.489908 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489856    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.489908 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489885    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-os-release\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.489908 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489894    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-sysconfig\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.489908 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489896    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-var-lib-kubelet\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.490158 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489929    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntrh9\" (UniqueName: \"kubernetes.io/projected/15cba5b8-1945-4d35-9762-b160c3d27bde-kube-api-access-ntrh9\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.490158 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.489971    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:46.490158 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093-iptables-alerter-script\") pod \"iptables-alerter-8dg7c\" (UID: \"cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093\") " pod="openshift-network-operator/iptables-alerter-8dg7c"
Jul 07 14:56:46.490158 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490032    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-sysctl-d\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.490158 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490059    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b78593d3-7844-497b-96a9-17cd45cb49aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.490675 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490649    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b78593d3-7844-497b-96a9-17cd45cb49aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.490768 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490686    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093-iptables-alerter-script\") pod \"iptables-alerter-8dg7c\" (UID: \"cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093\") " pod="openshift-network-operator/iptables-alerter-8dg7c"
Jul 07 14:56:46.490768 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490698    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-run-systemd\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.490875 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490785    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-etc-openvswitch\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.490875 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490792    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-sysctl-d\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.490875 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490813    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.490875 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490844    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-multus-cni-dir\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.491127 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490875    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9wtp\" (UniqueName: \"kubernetes.io/projected/b2a0a171-7c99-4f1b-b735-8f0083a4094b-kube-api-access-d9wtp\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.491127 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490913    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/feab3f31-3c5f-4cc5-abb6-cc0405cb862f-agent-certs\") pod \"konnectivity-agent-9zk5l\" (UID: \"feab3f31-3c5f-4cc5-abb6-cc0405cb862f\") " pod="kube-system/konnectivity-agent-9zk5l"
Jul 07 14:56:46.491127 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490938    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-cnibin\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.491127 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490964    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-multus-socket-dir-parent\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.491127 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.490988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-var-lib-cni-bin\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.491127 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491014    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-kubernetes\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.491127 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491038    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b02523f0-255e-4b20-8947-e3d553aa71c2-ovnkube-config\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.491127 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491069    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77pmh\" (UniqueName: \"kubernetes.io/projected/6aaa686c-76ef-4753-86e1-5df48442b8ce-kube-api-access-77pmh\") pod \"node-ca-ftfzh\" (UID: \"6aaa686c-76ef-4753-86e1-5df48442b8ce\") " pod="openshift-image-registry/node-ca-ftfzh"
Jul 07 14:56:46.491127 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491097    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-registration-dir\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.491127 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491123    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdmrs\" (UniqueName: \"kubernetes.io/projected/3418602d-7767-4f00-b89f-8903c6faf891-kube-api-access-hdmrs\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.491127 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491127    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-kubernetes\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktt92\" (UniqueName: \"kubernetes.io/projected/03e07f30-8257-4a4b-bb01-042bb672a64c-kube-api-access-ktt92\") pod \"node-resolver-cmwns\" (UID: \"03e07f30-8257-4a4b-bb01-042bb672a64c\") " pod="openshift-dns/node-resolver-cmwns"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491178    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/feab3f31-3c5f-4cc5-abb6-cc0405cb862f-konnectivity-ca\") pod \"konnectivity-agent-9zk5l\" (UID: \"feab3f31-3c5f-4cc5-abb6-cc0405cb862f\") " pod="kube-system/konnectivity-agent-9zk5l"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491202    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-run-netns\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491232    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-run-ovn\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491269    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/15cba5b8-1945-4d35-9762-b160c3d27bde-multus-daemon-config\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491263    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-registration-dir\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491294    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6aaa686c-76ef-4753-86e1-5df48442b8ce-host\") pod \"node-ca-ftfzh\" (UID: \"6aaa686c-76ef-4753-86e1-5df48442b8ce\") " pod="openshift-image-registry/node-ca-ftfzh"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491322    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b78593d3-7844-497b-96a9-17cd45cb49aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491375    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b02523f0-255e-4b20-8947-e3d553aa71c2-env-overrides\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491400    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b02523f0-255e-4b20-8947-e3d553aa71c2-ovnkube-script-lib\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-hostroot\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491451    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3418602d-7767-4f00-b89f-8903c6faf891-tmp\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491490    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-kubelet\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491552    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-node-log\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491594    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-log-socket\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491621    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qks96\" (UniqueName: \"kubernetes.io/projected/7fcdb256-eea5-428e-9492-2cd608fd8a23-kube-api-access-qks96\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:46.491712 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491650    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-run-openvswitch\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491693    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-host\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491747    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-etc-selinux\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491774    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-sys-fs\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491782    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491802    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52ksb\" (UniqueName: \"kubernetes.io/projected/b78593d3-7844-497b-96a9-17cd45cb49aa-kube-api-access-52ksb\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491832    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-cni-bin\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491858    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-cni-netd\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491884    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093-host-slash\") pod \"iptables-alerter-8dg7c\" (UID: \"cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093\") " pod="openshift-network-operator/iptables-alerter-8dg7c"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491909    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-modprobe-d\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491933    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-systemd\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491958    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3418602d-7767-4f00-b89f-8903c6faf891-etc-tuned\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.491982    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv656\" (UniqueName: \"kubernetes.io/projected/b02523f0-255e-4b20-8947-e3d553aa71c2-kube-api-access-cv656\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492011    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-run\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492039    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-var-lib-kubelet\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492067    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03e07f30-8257-4a4b-bb01-042bb672a64c-tmp-dir\") pod \"node-resolver-cmwns\" (UID: \"03e07f30-8257-4a4b-bb01-042bb672a64c\") " pod="openshift-dns/node-resolver-cmwns"
Jul 07 14:56:46.492571 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492094    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f99558a-08a5-4532-a131-eb30f2278bb5-kubelet-config\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492120    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b78593d3-7844-497b-96a9-17cd45cb49aa-os-release\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492144    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-systemd-units\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492172    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-var-lib-openvswitch\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-var-lib-cni-multus\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492225    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-multus-conf-dir\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492238    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-modprobe-d\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-lib-modules\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492280    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-sys\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492294    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-host\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492324    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-slash\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492353    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-run-k8s-cni-cncf-io\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492359    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093-host-slash\") pod \"iptables-alerter-8dg7c\" (UID: \"cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093\") " pod="openshift-network-operator/iptables-alerter-8dg7c"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492382    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-run-netns\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492437    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492454    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-systemd\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492489    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-etc-selinux\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.493468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492537    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-sys-fs\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492585    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-socket-dir\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492613    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-sysctl-conf\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492641    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wwxg\" (UniqueName: \"kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg\") pod \"network-check-target-5w2jq\" (UID: \"213be297-4d57-4440-9531-e4121fe87b19\") " pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492675    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-system-cni-dir\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492700    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15cba5b8-1945-4d35-9762-b160c3d27bde-cni-binary-copy\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492726    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03e07f30-8257-4a4b-bb01-042bb672a64c-hosts-file\") pod \"node-resolver-cmwns\" (UID: \"03e07f30-8257-4a4b-bb01-042bb672a64c\") " pod="openshift-dns/node-resolver-cmwns"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492779    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03e07f30-8257-4a4b-bb01-042bb672a64c-hosts-file\") pod \"node-resolver-cmwns\" (UID: \"03e07f30-8257-4a4b-bb01-042bb672a64c\") " pod="openshift-dns/node-resolver-cmwns"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492845    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-run\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492846    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-etc-sysctl-conf\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492870    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2a0a171-7c99-4f1b-b735-8f0083a4094b-socket-dir\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492887    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f99558a-08a5-4532-a131-eb30f2278bb5-kubelet-config\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492962    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-sys\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492985    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f99558a-08a5-4532-a131-eb30f2278bb5-dbus\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.492981    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3418602d-7767-4f00-b89f-8903c6faf891-lib-modules\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.493036    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.493068    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b78593d3-7844-497b-96a9-17cd45cb49aa-cnibin\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.494300 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.493092    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b02523f0-255e-4b20-8947-e3d553aa71c2-ovn-node-metrics-cert\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.497917 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.493101    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f99558a-08a5-4532-a131-eb30f2278bb5-dbus\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:46.497917 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.493113    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6aaa686c-76ef-4753-86e1-5df48442b8ce-serviceca\") pod \"node-ca-ftfzh\" (UID: \"6aaa686c-76ef-4753-86e1-5df48442b8ce\") " pod="openshift-image-registry/node-ca-ftfzh"
Jul 07 14:56:46.497917 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.493139    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-run-multus-certs\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.497917 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.493161    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-etc-kubernetes\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.497917 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.493178    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03e07f30-8257-4a4b-bb01-042bb672a64c-tmp-dir\") pod \"node-resolver-cmwns\" (UID: \"03e07f30-8257-4a4b-bb01-042bb672a64c\") " pod="openshift-dns/node-resolver-cmwns"
Jul 07 14:56:46.497917 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.493590    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:46.497917 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.493683    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret podName:5f99558a-08a5-4532-a131-eb30f2278bb5 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:46.993638652 +0000 UTC m=+3.006035191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret") pod "global-pull-secret-syncer-4rqtz" (UID: "5f99558a-08a5-4532-a131-eb30f2278bb5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:46.497917 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.497424    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3418602d-7767-4f00-b89f-8903c6faf891-tmp\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.499168 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.499143    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3418602d-7767-4f00-b89f-8903c6faf891-etc-tuned\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.500996 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.500975    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwlsl\" (UniqueName: \"kubernetes.io/projected/cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093-kube-api-access-qwlsl\") pod \"iptables-alerter-8dg7c\" (UID: \"cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093\") " pod="openshift-network-operator/iptables-alerter-8dg7c"
Jul 07 14:56:46.504068 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.504044    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdmrs\" (UniqueName: \"kubernetes.io/projected/3418602d-7767-4f00-b89f-8903c6faf891-kube-api-access-hdmrs\") pod \"tuned-5c8ng\" (UID: \"3418602d-7767-4f00-b89f-8903c6faf891\") " pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.506477 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.506454    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:46.506582 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.506482    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:46.506582 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.506496    2569 projected.go:196] Error preparing data for projected volume kube-api-access-7wwxg for pod openshift-network-diagnostics/network-check-target-5w2jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:46.506582 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.506560    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg podName:213be297-4d57-4440-9531-e4121fe87b19 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:47.00654135 +0000 UTC m=+3.018937891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7wwxg" (UniqueName: "kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg") pod "network-check-target-5w2jq" (UID: "213be297-4d57-4440-9531-e4121fe87b19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:46.509575 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.509548    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktt92\" (UniqueName: \"kubernetes.io/projected/03e07f30-8257-4a4b-bb01-042bb672a64c-kube-api-access-ktt92\") pod \"node-resolver-cmwns\" (UID: \"03e07f30-8257-4a4b-bb01-042bb672a64c\") " pod="openshift-dns/node-resolver-cmwns"
Jul 07 14:56:46.509960 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.509935    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9wtp\" (UniqueName: \"kubernetes.io/projected/b2a0a171-7c99-4f1b-b735-8f0083a4094b-kube-api-access-d9wtp\") pod \"aws-ebs-csi-driver-node-5t49k\" (UID: \"b2a0a171-7c99-4f1b-b735-8f0083a4094b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.516072 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.516029    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-158.ec2.internal" event={"ID":"831e9ec52a3238957c7f1fb3d14b894f","Type":"ContainerStarted","Data":"bdf927591819765c5f2bf3d9e4b91ddea4d46ad5d609535b35816f5458aef9c5"}
Jul 07 14:56:46.517368 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.517341    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal" event={"ID":"556c0bc2b69d90a68099396d7b905ebb","Type":"ContainerStarted","Data":"28ae783fc7f1e1bdff249616df1a5c2334103a95c9f8befab0090e4dedab452f"}
Jul 07 14:56:46.594210 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594128    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntrh9\" (UniqueName: \"kubernetes.io/projected/15cba5b8-1945-4d35-9762-b160c3d27bde-kube-api-access-ntrh9\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.594210 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594162    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:46.594210 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594181    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b78593d3-7844-497b-96a9-17cd45cb49aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.594210 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b78593d3-7844-497b-96a9-17cd45cb49aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.594564 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594231    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-run-systemd\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.594564 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594258    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-etc-openvswitch\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.594564 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594297    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.594564 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594308    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-run-systemd\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.594564 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594362    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-etc-openvswitch\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.594564 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594320    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-multus-cni-dir\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.594564 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594394    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.594564 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594401    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-multus-cni-dir\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.594564 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.594553    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594617    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/feab3f31-3c5f-4cc5-abb6-cc0405cb862f-agent-certs\") pod \"konnectivity-agent-9zk5l\" (UID: \"feab3f31-3c5f-4cc5-abb6-cc0405cb862f\") " pod="kube-system/konnectivity-agent-9zk5l"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.594627    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs podName:7fcdb256-eea5-428e-9492-2cd608fd8a23 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:47.094606994 +0000 UTC m=+3.107003541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs") pod "network-metrics-daemon-s5zzk" (UID: "7fcdb256-eea5-428e-9492-2cd608fd8a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594655    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-cnibin\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594685    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-multus-socket-dir-parent\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-var-lib-cni-bin\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594747    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b02523f0-255e-4b20-8947-e3d553aa71c2-ovnkube-config\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594787    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77pmh\" (UniqueName: \"kubernetes.io/projected/6aaa686c-76ef-4753-86e1-5df48442b8ce-kube-api-access-77pmh\") pod \"node-ca-ftfzh\" (UID: \"6aaa686c-76ef-4753-86e1-5df48442b8ce\") " pod="openshift-image-registry/node-ca-ftfzh"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/feab3f31-3c5f-4cc5-abb6-cc0405cb862f-konnectivity-ca\") pod \"konnectivity-agent-9zk5l\" (UID: \"feab3f31-3c5f-4cc5-abb6-cc0405cb862f\") " pod="kube-system/konnectivity-agent-9zk5l"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594830    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-cnibin\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594858    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b78593d3-7844-497b-96a9-17cd45cb49aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594891    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-multus-socket-dir-parent\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594935    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-var-lib-cni-bin\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594965    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-run-netns\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594992    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-run-ovn\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.594991    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b78593d3-7844-497b-96a9-17cd45cb49aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595017    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/15cba5b8-1945-4d35-9762-b160c3d27bde-multus-daemon-config\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.595081 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595040    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6aaa686c-76ef-4753-86e1-5df48442b8ce-host\") pod \"node-ca-ftfzh\" (UID: \"6aaa686c-76ef-4753-86e1-5df48442b8ce\") " pod="openshift-image-registry/node-ca-ftfzh"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595054    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-run-netns\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595066    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b78593d3-7844-497b-96a9-17cd45cb49aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595059    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-run-ovn\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595103    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b02523f0-255e-4b20-8947-e3d553aa71c2-env-overrides\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595127    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b02523f0-255e-4b20-8947-e3d553aa71c2-ovnkube-script-lib\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595131    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6aaa686c-76ef-4753-86e1-5df48442b8ce-host\") pod \"node-ca-ftfzh\" (UID: \"6aaa686c-76ef-4753-86e1-5df48442b8ce\") " pod="openshift-image-registry/node-ca-ftfzh"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595153    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-hostroot\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-kubelet\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595210    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-node-log\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595244    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-log-socket\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595272    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qks96\" (UniqueName: \"kubernetes.io/projected/7fcdb256-eea5-428e-9492-2cd608fd8a23-kube-api-access-qks96\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595320    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-run-openvswitch\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595350    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52ksb\" (UniqueName: \"kubernetes.io/projected/b78593d3-7844-497b-96a9-17cd45cb49aa-kube-api-access-52ksb\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595378    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-cni-bin\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595404    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-cni-netd\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595411    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/feab3f31-3c5f-4cc5-abb6-cc0405cb862f-konnectivity-ca\") pod \"konnectivity-agent-9zk5l\" (UID: \"feab3f31-3c5f-4cc5-abb6-cc0405cb862f\") " pod="kube-system/konnectivity-agent-9zk5l"
Jul 07 14:56:46.595967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595437    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv656\" (UniqueName: \"kubernetes.io/projected/b02523f0-255e-4b20-8947-e3d553aa71c2-kube-api-access-cv656\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595465    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-var-lib-kubelet\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595488    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b02523f0-255e-4b20-8947-e3d553aa71c2-ovnkube-config\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595495    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b78593d3-7844-497b-96a9-17cd45cb49aa-os-release\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595514    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-cni-bin\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595544    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-systemd-units\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595566    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b78593d3-7844-497b-96a9-17cd45cb49aa-os-release\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595576    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-var-lib-openvswitch\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595583    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-run-openvswitch\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595608    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-var-lib-cni-multus\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595628    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-var-lib-kubelet\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595634    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-multus-conf-dir\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595668    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-kubelet\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595677    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-node-log\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595689    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-multus-conf-dir\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595712    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-slash\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595738    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-systemd-units\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595788    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-run-k8s-cni-cncf-io\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.596877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595791    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-var-lib-openvswitch\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595811    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-var-lib-cni-multus\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595826    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-slash\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595832    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-cni-netd\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595741    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-run-k8s-cni-cncf-io\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595852    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-hostroot\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595854    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b02523f0-255e-4b20-8947-e3d553aa71c2-ovnkube-script-lib\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-run-netns\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595872    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-log-socket\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595881    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/15cba5b8-1945-4d35-9762-b160c3d27bde-multus-daemon-config\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595922    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-run-netns\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595930    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-system-cni-dir\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595955    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15cba5b8-1945-4d35-9762-b160c3d27bde-cni-binary-copy\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.595999    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b78593d3-7844-497b-96a9-17cd45cb49aa-cnibin\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596000    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-system-cni-dir\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596029    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b02523f0-255e-4b20-8947-e3d553aa71c2-ovn-node-metrics-cert\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596054    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6aaa686c-76ef-4753-86e1-5df48442b8ce-serviceca\") pod \"node-ca-ftfzh\" (UID: \"6aaa686c-76ef-4753-86e1-5df48442b8ce\") " pod="openshift-image-registry/node-ca-ftfzh"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596036    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b78593d3-7844-497b-96a9-17cd45cb49aa-cnibin\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.597716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596082    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-run-multus-certs\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596108    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-etc-kubernetes\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596136    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b78593d3-7844-497b-96a9-17cd45cb49aa-system-cni-dir\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596156    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b78593d3-7844-497b-96a9-17cd45cb49aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596204    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-etc-kubernetes\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596238    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596280    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-os-release\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596325    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b02523f0-255e-4b20-8947-e3d553aa71c2-env-overrides\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596340    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b78593d3-7844-497b-96a9-17cd45cb49aa-system-cni-dir\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596366    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-os-release\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596402    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b02523f0-255e-4b20-8947-e3d553aa71c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596407    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/15cba5b8-1945-4d35-9762-b160c3d27bde-host-run-multus-certs\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596654    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b78593d3-7844-497b-96a9-17cd45cb49aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596817    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b78593d3-7844-497b-96a9-17cd45cb49aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.596905    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15cba5b8-1945-4d35-9762-b160c3d27bde-cni-binary-copy\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.597067    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6aaa686c-76ef-4753-86e1-5df48442b8ce-serviceca\") pod \"node-ca-ftfzh\" (UID: \"6aaa686c-76ef-4753-86e1-5df48442b8ce\") " pod="openshift-image-registry/node-ca-ftfzh"
Jul 07 14:56:46.598500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.597882    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/feab3f31-3c5f-4cc5-abb6-cc0405cb862f-agent-certs\") pod \"konnectivity-agent-9zk5l\" (UID: \"feab3f31-3c5f-4cc5-abb6-cc0405cb862f\") " pod="kube-system/konnectivity-agent-9zk5l"
Jul 07 14:56:46.599129 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.598813    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b02523f0-255e-4b20-8947-e3d553aa71c2-ovn-node-metrics-cert\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.606803 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.606774    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77pmh\" (UniqueName: \"kubernetes.io/projected/6aaa686c-76ef-4753-86e1-5df48442b8ce-kube-api-access-77pmh\") pod \"node-ca-ftfzh\" (UID: \"6aaa686c-76ef-4753-86e1-5df48442b8ce\") " pod="openshift-image-registry/node-ca-ftfzh"
Jul 07 14:56:46.607053 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.607027    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv656\" (UniqueName: \"kubernetes.io/projected/b02523f0-255e-4b20-8947-e3d553aa71c2-kube-api-access-cv656\") pod \"ovnkube-node-gpxk2\" (UID: \"b02523f0-255e-4b20-8947-e3d553aa71c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.607216 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.607194    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntrh9\" (UniqueName: \"kubernetes.io/projected/15cba5b8-1945-4d35-9762-b160c3d27bde-kube-api-access-ntrh9\") pod \"multus-2xxrr\" (UID: \"15cba5b8-1945-4d35-9762-b160c3d27bde\") " pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.607687 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.607657    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qks96\" (UniqueName: \"kubernetes.io/projected/7fcdb256-eea5-428e-9492-2cd608fd8a23-kube-api-access-qks96\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:46.607967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.607946    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52ksb\" (UniqueName: \"kubernetes.io/projected/b78593d3-7844-497b-96a9-17cd45cb49aa-kube-api-access-52ksb\") pod \"multus-additional-cni-plugins-2nvp9\" (UID: \"b78593d3-7844-497b-96a9-17cd45cb49aa\") " pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.689755 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.689722    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8dg7c"
Jul 07 14:56:46.699431 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.699405    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cmwns"
Jul 07 14:56:46.707969 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.707947    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k"
Jul 07 14:56:46.723920 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.723897    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5c8ng"
Jul 07 14:56:46.731710 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.731689    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2nvp9"
Jul 07 14:56:46.742577 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.742557    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2xxrr"
Jul 07 14:56:46.747637 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.747618    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9zk5l"
Jul 07 14:56:46.756518 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.756499    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:56:46.761329 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.761305    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ftfzh"
Jul 07 14:56:46.999575 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:46.999479    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:46.999729 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.999613    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:46.999729 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:46.999686    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret podName:5f99558a-08a5-4532-a131-eb30f2278bb5 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:47.999666784 +0000 UTC m=+4.012063311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret") pod "global-pull-secret-syncer-4rqtz" (UID: "5f99558a-08a5-4532-a131-eb30f2278bb5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:47.100527 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.100484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wwxg\" (UniqueName: \"kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg\") pod \"network-check-target-5w2jq\" (UID: \"213be297-4d57-4440-9531-e4121fe87b19\") " pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:47.100736 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.100561    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:47.100736 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:47.100695    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:47.100736 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:47.100703    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:47.100736 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:47.100735    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:47.100927 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:47.100751    2569 projected.go:196] Error preparing data for projected volume kube-api-access-7wwxg for pod openshift-network-diagnostics/network-check-target-5w2jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:47.100927 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:47.100780    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs podName:7fcdb256-eea5-428e-9492-2cd608fd8a23 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:48.100760212 +0000 UTC m=+4.113156754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs") pod "network-metrics-daemon-s5zzk" (UID: "7fcdb256-eea5-428e-9492-2cd608fd8a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:47.100927 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:47.100808    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg podName:213be297-4d57-4440-9531-e4121fe87b19 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:48.100790894 +0000 UTC m=+4.113187437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7wwxg" (UniqueName: "kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg") pod "network-check-target-5w2jq" (UID: "213be297-4d57-4440-9531-e4121fe87b19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:47.435032 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.434924    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:51:47 +0000 UTC" deadline="2028-03-13 15:07:04.213436462 +0000 UTC"
Jul 07 14:56:47.435032 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.434982    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14760h10m16.778457076s"
Jul 07 14:56:47.490081 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:47.490047    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeab3f31_3c5f_4cc5_abb6_cc0405cb862f.slice/crio-b186f0549689d10573ae5ae0d7e2bbf40b83fff56066bd26f709e8f3e98112e3 WatchSource:0}: Error finding container b186f0549689d10573ae5ae0d7e2bbf40b83fff56066bd26f709e8f3e98112e3: Status 404 returned error can't find the container with id b186f0549689d10573ae5ae0d7e2bbf40b83fff56066bd26f709e8f3e98112e3
Jul 07 14:56:47.508954 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:47.508929    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78593d3_7844_497b_96a9_17cd45cb49aa.slice/crio-092dba9804dc34426c82e166fc04ef2371d8341704290886b80b49d514ba10e0 WatchSource:0}: Error finding container 092dba9804dc34426c82e166fc04ef2371d8341704290886b80b49d514ba10e0: Status 404 returned error can't find the container with id 092dba9804dc34426c82e166fc04ef2371d8341704290886b80b49d514ba10e0
Jul 07 14:56:47.509920 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:47.509891    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02523f0_255e_4b20_8947_e3d553aa71c2.slice/crio-da7268a75cf5454fda67df2980f6a85f60be2e180e7d658f610b5a724c55d0f3 WatchSource:0}: Error finding container da7268a75cf5454fda67df2980f6a85f60be2e180e7d658f610b5a724c55d0f3: Status 404 returned error can't find the container with id da7268a75cf5454fda67df2980f6a85f60be2e180e7d658f610b5a724c55d0f3
Jul 07 14:56:47.510626 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:47.510581    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3418602d_7767_4f00_b89f_8903c6faf891.slice/crio-26cb9c014bc3290da93920afcd4a8464f4eb8a92cef631fe5332ea4ed46c7a47 WatchSource:0}: Error finding container 26cb9c014bc3290da93920afcd4a8464f4eb8a92cef631fe5332ea4ed46c7a47: Status 404 returned error can't find the container with id 26cb9c014bc3290da93920afcd4a8464f4eb8a92cef631fe5332ea4ed46c7a47
Jul 07 14:56:47.511179 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.511158    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:47.511361 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:47.511332    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:56:47.511973 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:47.511839    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e07f30_8257_4a4b_bb01_042bb672a64c.slice/crio-2d7f99c0ffe8a1725707e9acb4cbac5957242b254c2b9346c98a58df00038376 WatchSource:0}: Error finding container 2d7f99c0ffe8a1725707e9acb4cbac5957242b254c2b9346c98a58df00038376: Status 404 returned error can't find the container with id 2d7f99c0ffe8a1725707e9acb4cbac5957242b254c2b9346c98a58df00038376
Jul 07 14:56:47.512689 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:47.512663    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15cba5b8_1945_4d35_9762_b160c3d27bde.slice/crio-190cc31476479c4afa5217bdec8d3b83dc5678faef0d3608ad967c859239c2cf WatchSource:0}: Error finding container 190cc31476479c4afa5217bdec8d3b83dc5678faef0d3608ad967c859239c2cf: Status 404 returned error can't find the container with id 190cc31476479c4afa5217bdec8d3b83dc5678faef0d3608ad967c859239c2cf
Jul 07 14:56:47.513806 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:47.513705    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aaa686c_76ef_4753_86e1_5df48442b8ce.slice/crio-f977d4d665c801955cb7c9fae12cdb5ce68665df29d125f442a1eff45106c233 WatchSource:0}: Error finding container f977d4d665c801955cb7c9fae12cdb5ce68665df29d125f442a1eff45106c233: Status 404 returned error can't find the container with id f977d4d665c801955cb7c9fae12cdb5ce68665df29d125f442a1eff45106c233
Jul 07 14:56:47.515499 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:47.515380    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7be83c_ef1c_4e9c_9940_5c2cfe0fa093.slice/crio-fc8e44a620852ef93ecee18e130e518d10413fbb40bb988f384f4773a6542c6d WatchSource:0}: Error finding container fc8e44a620852ef93ecee18e130e518d10413fbb40bb988f384f4773a6542c6d: Status 404 returned error can't find the container with id fc8e44a620852ef93ecee18e130e518d10413fbb40bb988f384f4773a6542c6d
Jul 07 14:56:47.517000 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:56:47.516979    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2a0a171_7c99_4f1b_b735_8f0083a4094b.slice/crio-aad2a0b6e6ac687b5797d79d6cafa62244938b601cb973c95b21a375325d78b5 WatchSource:0}: Error finding container aad2a0b6e6ac687b5797d79d6cafa62244938b601cb973c95b21a375325d78b5: Status 404 returned error can't find the container with id aad2a0b6e6ac687b5797d79d6cafa62244938b601cb973c95b21a375325d78b5
Jul 07 14:56:47.522690 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.522611    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2nvp9" event={"ID":"b78593d3-7844-497b-96a9-17cd45cb49aa","Type":"ContainerStarted","Data":"092dba9804dc34426c82e166fc04ef2371d8341704290886b80b49d514ba10e0"}
Jul 07 14:56:47.524635 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.524602    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9zk5l" event={"ID":"feab3f31-3c5f-4cc5-abb6-cc0405cb862f","Type":"ContainerStarted","Data":"b186f0549689d10573ae5ae0d7e2bbf40b83fff56066bd26f709e8f3e98112e3"}
Jul 07 14:56:47.526084 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.526057    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8dg7c" event={"ID":"cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093","Type":"ContainerStarted","Data":"fc8e44a620852ef93ecee18e130e518d10413fbb40bb988f384f4773a6542c6d"}
Jul 07 14:56:47.527420 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.527394    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ftfzh" event={"ID":"6aaa686c-76ef-4753-86e1-5df48442b8ce","Type":"ContainerStarted","Data":"f977d4d665c801955cb7c9fae12cdb5ce68665df29d125f442a1eff45106c233"}
Jul 07 14:56:47.528650 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.528627    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2xxrr" event={"ID":"15cba5b8-1945-4d35-9762-b160c3d27bde","Type":"ContainerStarted","Data":"190cc31476479c4afa5217bdec8d3b83dc5678faef0d3608ad967c859239c2cf"}
Jul 07 14:56:47.529830 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.529814    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5c8ng" event={"ID":"3418602d-7767-4f00-b89f-8903c6faf891","Type":"ContainerStarted","Data":"26cb9c014bc3290da93920afcd4a8464f4eb8a92cef631fe5332ea4ed46c7a47"}
Jul 07 14:56:47.531164 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.531137    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2" event={"ID":"b02523f0-255e-4b20-8947-e3d553aa71c2","Type":"ContainerStarted","Data":"da7268a75cf5454fda67df2980f6a85f60be2e180e7d658f610b5a724c55d0f3"}
Jul 07 14:56:47.532185 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:47.532163    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cmwns" event={"ID":"03e07f30-8257-4a4b-bb01-042bb672a64c","Type":"ContainerStarted","Data":"2d7f99c0ffe8a1725707e9acb4cbac5957242b254c2b9346c98a58df00038376"}
Jul 07 14:56:48.011820 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:48.011118    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:48.011820 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:48.011312    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:48.011820 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:48.011397    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret podName:5f99558a-08a5-4532-a131-eb30f2278bb5 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:50.01137839 +0000 UTC m=+6.023774919 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret") pod "global-pull-secret-syncer-4rqtz" (UID: "5f99558a-08a5-4532-a131-eb30f2278bb5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:48.112027 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:48.111985    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wwxg\" (UniqueName: \"kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg\") pod \"network-check-target-5w2jq\" (UID: \"213be297-4d57-4440-9531-e4121fe87b19\") " pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:48.112193 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:48.112068    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:48.112253 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:48.112197    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:48.112307 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:48.112262    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs podName:7fcdb256-eea5-428e-9492-2cd608fd8a23 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:50.112241889 +0000 UTC m=+6.124638421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs") pod "network-metrics-daemon-s5zzk" (UID: "7fcdb256-eea5-428e-9492-2cd608fd8a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:48.112768 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:48.112747    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:48.112848 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:48.112774    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:48.112848 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:48.112789    2569 projected.go:196] Error preparing data for projected volume kube-api-access-7wwxg for pod openshift-network-diagnostics/network-check-target-5w2jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:48.112848 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:48.112838    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg podName:213be297-4d57-4440-9531-e4121fe87b19 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:50.112820667 +0000 UTC m=+6.125217210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7wwxg" (UniqueName: "kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg") pod "network-check-target-5w2jq" (UID: "213be297-4d57-4440-9531-e4121fe87b19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:48.435414 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:48.435260    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:51:47 +0000 UTC" deadline="2028-04-14 11:19:45.486786408 +0000 UTC"
Jul 07 14:56:48.435414 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:48.435380    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15524h22m57.05141219s"
Jul 07 14:56:48.521388 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:48.521314    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:48.521552 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:48.521464    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:56:48.521906 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:48.521881    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:48.522026 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:48.522005    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:56:48.546267 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:48.546234    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-158.ec2.internal" event={"ID":"831e9ec52a3238957c7f1fb3d14b894f","Type":"ContainerStarted","Data":"86ddf757693eee5799e83f51bf6116b670ac55796163f5999935f70ca1ab2550"}
Jul 07 14:56:48.559881 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:48.559851    2569 generic.go:356] "Generic (PLEG): container finished" podID="556c0bc2b69d90a68099396d7b905ebb" containerID="9e56f0114bd871e93080f2c4de01b60ad2aa0cc3c656b09d2e460622b69403c9" exitCode=0
Jul 07 14:56:48.560006 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:48.559889    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal" event={"ID":"556c0bc2b69d90a68099396d7b905ebb","Type":"ContainerDied","Data":"9e56f0114bd871e93080f2c4de01b60ad2aa0cc3c656b09d2e460622b69403c9"}
Jul 07 14:56:48.574538 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:48.574488    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-158.ec2.internal" podStartSLOduration=2.574469289 podStartE2EDuration="2.574469289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:48.574257406 +0000 UTC m=+4.586653984" watchObservedRunningTime="2026-07-07 14:56:48.574469289 +0000 UTC m=+4.586865840"
Jul 07 14:56:48.579076 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:48.579048    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k" event={"ID":"b2a0a171-7c99-4f1b-b735-8f0083a4094b","Type":"ContainerStarted","Data":"aad2a0b6e6ac687b5797d79d6cafa62244938b601cb973c95b21a375325d78b5"}
Jul 07 14:56:49.511677 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:49.511642    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:49.512167 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:49.511784    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:56:49.595503 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:49.595418    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal" event={"ID":"556c0bc2b69d90a68099396d7b905ebb","Type":"ContainerStarted","Data":"18a49ec4fc3e77af90489ede3b812dbe34b4da31e94995cc7da1aee073186096"}
Jul 07 14:56:50.031634 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:50.030939    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:50.031634 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:50.031197    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:50.031634 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:50.031264    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret podName:5f99558a-08a5-4532-a131-eb30f2278bb5 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:54.031246004 +0000 UTC m=+10.043642535 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret") pod "global-pull-secret-syncer-4rqtz" (UID: "5f99558a-08a5-4532-a131-eb30f2278bb5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:50.132446 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:50.131626    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wwxg\" (UniqueName: \"kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg\") pod \"network-check-target-5w2jq\" (UID: \"213be297-4d57-4440-9531-e4121fe87b19\") " pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:50.132446 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:50.131702    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:50.132446 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:50.131841    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:50.132446 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:50.131909    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs podName:7fcdb256-eea5-428e-9492-2cd608fd8a23 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:54.131889107 +0000 UTC m=+10.144285650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs") pod "network-metrics-daemon-s5zzk" (UID: "7fcdb256-eea5-428e-9492-2cd608fd8a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:50.132446 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:50.132004    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:50.132446 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:50.132023    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:50.132446 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:50.132036    2569 projected.go:196] Error preparing data for projected volume kube-api-access-7wwxg for pod openshift-network-diagnostics/network-check-target-5w2jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:50.132446 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:50.132082    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg podName:213be297-4d57-4440-9531-e4121fe87b19 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:54.132060977 +0000 UTC m=+10.144457504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7wwxg" (UniqueName: "kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg") pod "network-check-target-5w2jq" (UID: "213be297-4d57-4440-9531-e4121fe87b19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:50.512676 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:50.512568    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:50.513153 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:50.512720    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:56:50.513219 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:50.513158    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:50.513312 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:50.513276    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:56:51.511623 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:51.511583    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:51.511841 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:51.511739    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:56:52.512793 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:52.512705    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:52.513264 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:52.512855    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:56:52.513451 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:52.513426    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:52.513587 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:52.513562    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:56:53.511388 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:53.511347    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:53.511609 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:53.511495    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:56:54.065570 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:54.065526    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:54.066096 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:54.065712    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:54.066096 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:54.065799    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret podName:5f99558a-08a5-4532-a131-eb30f2278bb5 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:02.065776764 +0000 UTC m=+18.078173296 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret") pod "global-pull-secret-syncer-4rqtz" (UID: "5f99558a-08a5-4532-a131-eb30f2278bb5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:56:54.166934 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:54.166893    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wwxg\" (UniqueName: \"kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg\") pod \"network-check-target-5w2jq\" (UID: \"213be297-4d57-4440-9531-e4121fe87b19\") " pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:54.167124 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:54.166980    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:54.167124 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:54.167031    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:56:54.167124 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:54.167057    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:56:54.167124 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:54.167069    2569 projected.go:196] Error preparing data for projected volume kube-api-access-7wwxg for pod openshift-network-diagnostics/network-check-target-5w2jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:54.167124 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:54.167083    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:54.167124 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:54.167127    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg podName:213be297-4d57-4440-9531-e4121fe87b19 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:02.167112925 +0000 UTC m=+18.179509452 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7wwxg" (UniqueName: "kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg") pod "network-check-target-5w2jq" (UID: "213be297-4d57-4440-9531-e4121fe87b19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:56:54.167461 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:54.167144    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs podName:7fcdb256-eea5-428e-9492-2cd608fd8a23 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:02.167137114 +0000 UTC m=+18.179533640 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs") pod "network-metrics-daemon-s5zzk" (UID: "7fcdb256-eea5-428e-9492-2cd608fd8a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:56:54.512756 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:54.512662    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:54.512945 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:54.512771    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:56:54.512945 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:54.512920    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:54.513074 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:54.513035    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:56:55.511564 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:55.511527    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:55.512181 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:55.511673    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:56:56.511375 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:56.511335    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:56.511572 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:56.511334    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:56.511572 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:56.511505    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:56:56.511572 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:56.511545    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:56:57.512166 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:57.512131    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:57.512712 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:57.512266    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:56:58.511334 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:58.511299    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:56:58.511510 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:58.511340    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:56:58.511510 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:58.511436    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:56:58.511626 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:58.511540    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:56:59.511155 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:56:59.511120    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:56:59.511588 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:56:59.511276    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:57:00.511411 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:00.511376    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:00.511851 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:00.511522    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:57:00.511851 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:00.511583    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:00.511851 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:00.511705    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:57:01.511502 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:01.511471    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:01.512007 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:01.511577    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:57:02.130609 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:02.130573    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:02.130797 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:02.130723    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:57:02.130797 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:02.130792    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret podName:5f99558a-08a5-4532-a131-eb30f2278bb5 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:18.13077273 +0000 UTC m=+34.143169256 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret") pod "global-pull-secret-syncer-4rqtz" (UID: "5f99558a-08a5-4532-a131-eb30f2278bb5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:57:02.231488 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:02.231450    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wwxg\" (UniqueName: \"kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg\") pod \"network-check-target-5w2jq\" (UID: \"213be297-4d57-4440-9531-e4121fe87b19\") " pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:02.231642 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:02.231522    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:02.231642 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:02.231627    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:57:02.231748 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:02.231638    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:57:02.231748 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:02.231666    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:57:02.231748 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:02.231680    2569 projected.go:196] Error preparing data for projected volume kube-api-access-7wwxg for pod openshift-network-diagnostics/network-check-target-5w2jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:57:02.231748 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:02.231688    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs podName:7fcdb256-eea5-428e-9492-2cd608fd8a23 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:18.231669403 +0000 UTC m=+34.244065927 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs") pod "network-metrics-daemon-s5zzk" (UID: "7fcdb256-eea5-428e-9492-2cd608fd8a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:57:02.231748 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:02.231729    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg podName:213be297-4d57-4440-9531-e4121fe87b19 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:18.231711885 +0000 UTC m=+34.244108453 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7wwxg" (UniqueName: "kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg") pod "network-check-target-5w2jq" (UID: "213be297-4d57-4440-9531-e4121fe87b19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:57:02.512291 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:02.512187    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:02.512725 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:02.512195    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:02.512725 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:02.512354    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:57:02.512725 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:02.512398    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:57:03.512206 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:03.512126    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:03.512397 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:03.512242    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:57:04.512580 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:04.512537    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:04.513033 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:04.512663    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:57:04.513033 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:04.512674    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:04.513033 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:04.512790    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:57:05.512128 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:05.512097    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:05.512350 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:05.512220    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:57:06.512183 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:06.512147    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:06.512717 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:06.512198    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:06.512717 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:06.512311    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:57:06.512717 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:06.512452    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:57:07.511957 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.511644    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:07.512173 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:07.512037    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:57:07.633121 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.633049    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cmwns" event={"ID":"03e07f30-8257-4a4b-bb01-042bb672a64c","Type":"ContainerStarted","Data":"763361e95f425dbea9e093c6488c41795e106f080b1edebcd2ce85f4be1d2ed6"}
Jul 07 14:57:07.634515 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.634493    2569 generic.go:356] "Generic (PLEG): container finished" podID="b78593d3-7844-497b-96a9-17cd45cb49aa" containerID="4dc7ad79e58ac92a556ad6b42b57acd4b5e80a0d29a4ad8bec906b959b3062cd" exitCode=0
Jul 07 14:57:07.634609 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.634558    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2nvp9" event={"ID":"b78593d3-7844-497b-96a9-17cd45cb49aa","Type":"ContainerDied","Data":"4dc7ad79e58ac92a556ad6b42b57acd4b5e80a0d29a4ad8bec906b959b3062cd"}
Jul 07 14:57:07.635959 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.635895    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9zk5l" event={"ID":"feab3f31-3c5f-4cc5-abb6-cc0405cb862f","Type":"ContainerStarted","Data":"4236460e104814222e4eafc1130664d4585583378ce63681b8cc5297791ff071"}
Jul 07 14:57:07.637381 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.637356    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ftfzh" event={"ID":"6aaa686c-76ef-4753-86e1-5df48442b8ce","Type":"ContainerStarted","Data":"c43b634f3d441d1864adc2cf7fc7d30770d694d53e7afcd1d93b6071e726b02e"}
Jul 07 14:57:07.638721 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.638691    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2xxrr" event={"ID":"15cba5b8-1945-4d35-9762-b160c3d27bde","Type":"ContainerStarted","Data":"154345ad968e3a9b2f931d62890ffbff45a7443c83547fc92f6d00a16b97dd7f"}
Jul 07 14:57:07.640036 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.640017    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5c8ng" event={"ID":"3418602d-7767-4f00-b89f-8903c6faf891","Type":"ContainerStarted","Data":"3648d5bac6e14652e2f324bb1bf59c40f95517b3ca1eeb9c7ab1ccee0cb73a27"}
Jul 07 14:57:07.642779 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.642757    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2" event={"ID":"b02523f0-255e-4b20-8947-e3d553aa71c2","Type":"ContainerStarted","Data":"aebe96d6953a95b02bc7d93ec7a32a44382764df1ffab445b876919ebd65bde2"}
Jul 07 14:57:07.642863 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.642830    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2" event={"ID":"b02523f0-255e-4b20-8947-e3d553aa71c2","Type":"ContainerStarted","Data":"018fad7329dc8e1b920d87333b048678ff9a1600b1f67417f758a8b2a4357e1f"}
Jul 07 14:57:07.642905 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.642881    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2" event={"ID":"b02523f0-255e-4b20-8947-e3d553aa71c2","Type":"ContainerStarted","Data":"999cb785274d6afa8baccffcc8e095c061a5846476a87e11e91230a30c56740b"}
Jul 07 14:57:07.642945 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.642923    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2" event={"ID":"b02523f0-255e-4b20-8947-e3d553aa71c2","Type":"ContainerStarted","Data":"85be16b6331351792b1198eade3efabdfd67bccf9933fea2212c56aa81b9ff2d"}
Jul 07 14:57:07.642993 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.642957    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2" event={"ID":"b02523f0-255e-4b20-8947-e3d553aa71c2","Type":"ContainerStarted","Data":"ba048d7694014720298149c86d74cdf0aeb6dc52663ca23daff727434df2c4e8"}
Jul 07 14:57:07.643049 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.642994    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2" event={"ID":"b02523f0-255e-4b20-8947-e3d553aa71c2","Type":"ContainerStarted","Data":"a5e612303ef0e8f5549ea723a3dd18b1d1a993cf7a0ce8cc16e5467d7ee16628"}
Jul 07 14:57:07.643972 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.643955    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k" event={"ID":"b2a0a171-7c99-4f1b-b735-8f0083a4094b","Type":"ContainerStarted","Data":"b3a6306aabaf0f1cca05481981b30dae1697d568ef7e5293b86376252f96ffdc"}
Jul 07 14:57:07.649804 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.649757    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cmwns" podStartSLOduration=3.386336648 podStartE2EDuration="22.649743067s" totalImagesPullingTime="19.263406419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:07.648333995 +0000 UTC m=+23.660730552" watchObservedRunningTime="2026-07-07 14:57:07.649743067 +0000 UTC m=+23.662139614"
Jul 07 14:57:07.649892 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.649847    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-158.ec2.internal" podStartSLOduration=21.649841715 podStartE2EDuration="21.649841715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:49.615661155 +0000 UTC m=+5.628057705" watchObservedRunningTime="2026-07-07 14:57:07.649841715 +0000 UTC m=+23.662238261"
Jul 07 14:57:07.665805 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.665770    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5c8ng" podStartSLOduration=3.401844848 podStartE2EDuration="22.66576229s" totalImagesPullingTime="19.263917442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:07.66544227 +0000 UTC m=+23.677838815" watchObservedRunningTime="2026-07-07 14:57:07.66576229 +0000 UTC m=+23.678158835"
Jul 07 14:57:07.684386 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.684333    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2xxrr" podStartSLOduration=3.359819231 podStartE2EDuration="22.684317646s" totalImagesPullingTime="19.324498415s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:07.684185541 +0000 UTC m=+23.696582115" watchObservedRunningTime="2026-07-07 14:57:07.684317646 +0000 UTC m=+23.696714193"
Jul 07 14:57:07.717560 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.717527    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9zk5l"
Jul 07 14:57:07.718161 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.718136    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9zk5l"
Jul 07 14:57:07.739397 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.739347    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9zk5l" podStartSLOduration=11.336351123 podStartE2EDuration="22.739331577s" totalImagesPullingTime="11.402980454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:07.70580662 +0000 UTC m=+23.718203165" watchObservedRunningTime="2026-07-07 14:57:07.739331577 +0000 UTC m=+23.751728122"
Jul 07 14:57:07.797559 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.797510    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ftfzh" podStartSLOduration=3.536297789 podStartE2EDuration="22.797492138s" totalImagesPullingTime="19.261194349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:07.766369153 +0000 UTC m=+23.778765725" watchObservedRunningTime="2026-07-07 14:57:07.797492138 +0000 UTC m=+23.809888684"
Jul 07 14:57:07.835914 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.835878    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:07.909417 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:07.909340    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:08.219412 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.219368    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 14:57:08.449157 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.448981    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T14:57:08.219387011Z","UUID":"7ad7e308-bc1c-410a-9f57-3f9c1351968f","Handler":null,"Name":"","Endpoint":""}
Jul 07 14:57:08.450625 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.450600    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 14:57:08.450754 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.450635    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 14:57:08.511848 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.511820    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:08.512088 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:08.511974    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:57:08.512088 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.511990    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:08.512214 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:08.512105    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:57:08.647929 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.647891    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8dg7c" event={"ID":"cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093","Type":"ContainerStarted","Data":"057faca958aa08104b95736a02a5d6c1c96a49e2813a63a96daf0bb53de46f57"}
Jul 07 14:57:08.650187 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.650163    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k" event={"ID":"b2a0a171-7c99-4f1b-b735-8f0083a4094b","Type":"ContainerStarted","Data":"80cc72c3a387eaf3446065840ea79d20bfd85509e15d9804cd25ae050f6f2a44"}
Jul 07 14:57:08.650948 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.650928    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9zk5l"
Jul 07 14:57:08.651082 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.650989    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9zk5l"
Jul 07 14:57:08.694678 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.694625    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8dg7c" podStartSLOduration=4.434791409 podStartE2EDuration="23.694604189s" totalImagesPullingTime="19.25981278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:08.670044631 +0000 UTC m=+24.682441179" watchObservedRunningTime="2026-07-07 14:57:08.694604189 +0000 UTC m=+24.707000736"
Jul 07 14:57:08.736135 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.735976    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:08.815900 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.815822    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:08.907337 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:08.907302    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:09.181728 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:09.181692    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:09.282713 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:09.282679    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:09.511156 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:09.511133    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:09.511357 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:09.511325    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:57:09.656310 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:09.656276    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2" event={"ID":"b02523f0-255e-4b20-8947-e3d553aa71c2","Type":"ContainerStarted","Data":"ef1ccfcd1a0b190b71ed41483851b83c58568dd80bf912c534008ff56c7f741a"}
Jul 07 14:57:09.658940 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:09.658812    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k" event={"ID":"b2a0a171-7c99-4f1b-b735-8f0083a4094b","Type":"ContainerStarted","Data":"5f03d4cf64bc4093c26f18ac6c9bca722d1be8937bea1b392333c644c7358ec2"}
Jul 07 14:57:10.511580 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:10.511511    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:10.511844 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:10.511656    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:57:10.511844 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:10.511688    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:10.511844 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:10.511817    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:57:11.512158 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:11.511938    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:11.512621 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:11.512287    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:57:12.511614 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:12.511427    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:12.511780 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:12.511449    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:12.511780 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:12.511708    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:57:12.511858 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:12.511774    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:57:12.671840 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:12.671803    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2" event={"ID":"b02523f0-255e-4b20-8947-e3d553aa71c2","Type":"ContainerStarted","Data":"be93c9200fddee8ec09282f9b19c9d1e88baff3232107b4a10b7dbe60760196e"}
Jul 07 14:57:12.672691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:12.672073    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:57:12.673737 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:12.673709    2569 generic.go:356] "Generic (PLEG): container finished" podID="b78593d3-7844-497b-96a9-17cd45cb49aa" containerID="520392582e0cb5b559a7107f92df89c59dce902ae5fc5cc6b068a56b36c8b418" exitCode=0
Jul 07 14:57:12.673876 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:12.673753    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2nvp9" event={"ID":"b78593d3-7844-497b-96a9-17cd45cb49aa","Type":"ContainerDied","Data":"520392582e0cb5b559a7107f92df89c59dce902ae5fc5cc6b068a56b36c8b418"}
Jul 07 14:57:12.688232 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:12.688214    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:57:12.703336 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:12.703297    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2" podStartSLOduration=8.378726612 podStartE2EDuration="27.703282978s" totalImagesPullingTime="19.324556366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:12.702048786 +0000 UTC m=+28.714445357" watchObservedRunningTime="2026-07-07 14:57:12.703282978 +0000 UTC m=+28.715679523"
Jul 07 14:57:12.703500 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:12.703483    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5t49k" podStartSLOduration=6.246867771 podStartE2EDuration="27.703475773s" totalImagesPullingTime="21.456608002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:09.683209683 +0000 UTC m=+25.695606253" watchObservedRunningTime="2026-07-07 14:57:12.703475773 +0000 UTC m=+28.715872364"
Jul 07 14:57:12.868406 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:12.868320    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:57:13.511743 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:13.511710    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:13.511946 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:13.511826    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:57:13.676738 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:13.676705    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:57:13.688991 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:13.688955    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5w2jq"]
Jul 07 14:57:13.689164 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:13.689082    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:13.689224 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:13.689181    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:57:13.689689 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:13.689670    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s5zzk"]
Jul 07 14:57:13.689804 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:13.689764    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:13.689863 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:13.689850    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:57:13.690349 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:13.690327    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4rqtz"]
Jul 07 14:57:13.690460 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:13.690436    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:13.690550 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:13.690532    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:57:13.693850 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:13.693828    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:57:14.680313 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:14.680102    2569 generic.go:356] "Generic (PLEG): container finished" podID="b78593d3-7844-497b-96a9-17cd45cb49aa" containerID="80026bb008476dadf8a6d722bf40ce70766de3ad30865a364e1fd1a486b1d518" exitCode=0
Jul 07 14:57:14.680313 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:14.680178    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2nvp9" event={"ID":"b78593d3-7844-497b-96a9-17cd45cb49aa","Type":"ContainerDied","Data":"80026bb008476dadf8a6d722bf40ce70766de3ad30865a364e1fd1a486b1d518"}
Jul 07 14:57:15.511973 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:15.511936    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:15.512144 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:15.511936    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:15.512144 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:15.512047    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:57:15.512144 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:15.511936    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:15.512485 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:15.512108    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:57:15.512485 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:15.512182    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:57:16.686419 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:16.686379    2569 generic.go:356] "Generic (PLEG): container finished" podID="b78593d3-7844-497b-96a9-17cd45cb49aa" containerID="c4256d9c12b0799f8e0cb056946496f109a2da2721c9b1dd2762f0c487dfaef3" exitCode=0
Jul 07 14:57:16.686888 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:16.686424    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2nvp9" event={"ID":"b78593d3-7844-497b-96a9-17cd45cb49aa","Type":"ContainerDied","Data":"c4256d9c12b0799f8e0cb056946496f109a2da2721c9b1dd2762f0c487dfaef3"}
Jul 07 14:57:17.512021 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:17.511982    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:17.512214 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:17.512100    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:17.512214 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:17.512108    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:57:17.512214 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:17.512136    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:17.512411 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:17.512244    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:57:17.512411 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:17.512341    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:57:18.151941 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:18.151912    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:18.152367 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:18.152045    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:57:18.152367 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:18.152095    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret podName:5f99558a-08a5-4532-a131-eb30f2278bb5 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:50.152082954 +0000 UTC m=+66.164479477 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret") pod "global-pull-secret-syncer-4rqtz" (UID: "5f99558a-08a5-4532-a131-eb30f2278bb5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:57:18.252349 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:18.252312    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wwxg\" (UniqueName: \"kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg\") pod \"network-check-target-5w2jq\" (UID: \"213be297-4d57-4440-9531-e4121fe87b19\") " pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:18.252513 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:18.252385    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:18.252513 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:18.252488    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:57:18.252604 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:18.252563    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs podName:7fcdb256-eea5-428e-9492-2cd608fd8a23 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:50.252542316 +0000 UTC m=+66.264938846 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs") pod "network-metrics-daemon-s5zzk" (UID: "7fcdb256-eea5-428e-9492-2cd608fd8a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:57:18.252604 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:18.252497    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:57:18.252604 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:18.252595    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:57:18.252760 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:18.252610    2569 projected.go:196] Error preparing data for projected volume kube-api-access-7wwxg for pod openshift-network-diagnostics/network-check-target-5w2jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:57:18.252760 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:18.252664    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg podName:213be297-4d57-4440-9531-e4121fe87b19 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:50.252648369 +0000 UTC m=+66.265044908 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7wwxg" (UniqueName: "kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg") pod "network-check-target-5w2jq" (UID: "213be297-4d57-4440-9531-e4121fe87b19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:57:18.379261 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:18.379224    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:18.475700 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:18.475653    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:19.511679 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:19.511643    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:19.512365 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:19.511642    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:19.512365 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:19.511771    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5w2jq" podUID="213be297-4d57-4440-9531-e4121fe87b19"
Jul 07 14:57:19.512365 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:19.511642    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:19.512365 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:19.511845    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4rqtz" podUID="5f99558a-08a5-4532-a131-eb30f2278bb5"
Jul 07 14:57:19.512365 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:19.511937    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5zzk" podUID="7fcdb256-eea5-428e-9492-2cd608fd8a23"
Jul 07 14:57:20.805158 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.805131    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-158.ec2.internal" event="NodeReady"
Jul 07 14:57:20.805504 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.805335    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 14:57:20.865367 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.865341    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wxktg"]
Jul 07 14:57:20.868511 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.868491    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mvtdl"]
Jul 07 14:57:20.868720 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.868623    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:20.870110 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.870094    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mvtdl"
Jul 07 14:57:20.871353 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.871262    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 14:57:20.871353 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.871306    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pkm7x\""
Jul 07 14:57:20.871585 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.871500    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 14:57:20.874164 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.874145    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 14:57:20.874325 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.874306    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 14:57:20.875330 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.875309    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bbd6b\""
Jul 07 14:57:20.875431 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.875312    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 14:57:20.888371 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.888322    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wxktg"]
Jul 07 14:57:20.891160 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.891041    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mvtdl"]
Jul 07 14:57:20.971897 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.971831    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5c5d52e-2d2c-429a-97e9-752108e3d730-cert\") pod \"ingress-canary-mvtdl\" (UID: \"e5c5d52e-2d2c-429a-97e9-752108e3d730\") " pod="openshift-ingress-canary/ingress-canary-mvtdl"
Jul 07 14:57:20.971897 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.971872    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7dcp\" (UniqueName: \"kubernetes.io/projected/aafa76ab-df1f-4aba-9736-16236535b661-kube-api-access-l7dcp\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:20.971897 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.971892    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:20.972086 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.971935    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aafa76ab-df1f-4aba-9736-16236535b661-tmp-dir\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:20.972086 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.971977    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv7qk\" (UniqueName: \"kubernetes.io/projected/e5c5d52e-2d2c-429a-97e9-752108e3d730-kube-api-access-cv7qk\") pod \"ingress-canary-mvtdl\" (UID: \"e5c5d52e-2d2c-429a-97e9-752108e3d730\") " pod="openshift-ingress-canary/ingress-canary-mvtdl"
Jul 07 14:57:20.972086 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:20.971995    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aafa76ab-df1f-4aba-9736-16236535b661-config-volume\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:21.035929 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.035899    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:21.072821 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.072784    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7qk\" (UniqueName: \"kubernetes.io/projected/e5c5d52e-2d2c-429a-97e9-752108e3d730-kube-api-access-cv7qk\") pod \"ingress-canary-mvtdl\" (UID: \"e5c5d52e-2d2c-429a-97e9-752108e3d730\") " pod="openshift-ingress-canary/ingress-canary-mvtdl"
Jul 07 14:57:21.072821 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.072818    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aafa76ab-df1f-4aba-9736-16236535b661-config-volume\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:21.073010 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.072845    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5c5d52e-2d2c-429a-97e9-752108e3d730-cert\") pod \"ingress-canary-mvtdl\" (UID: \"e5c5d52e-2d2c-429a-97e9-752108e3d730\") " pod="openshift-ingress-canary/ingress-canary-mvtdl"
Jul 07 14:57:21.073010 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.072954    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7dcp\" (UniqueName: \"kubernetes.io/projected/aafa76ab-df1f-4aba-9736-16236535b661-kube-api-access-l7dcp\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:21.073010 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.072992    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:21.073152 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.073036    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aafa76ab-df1f-4aba-9736-16236535b661-tmp-dir\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:21.073152 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:21.073073    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:21.073152 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:21.073134    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls podName:aafa76ab-df1f-4aba-9736-16236535b661 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:21.57311359 +0000 UTC m=+37.585510132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls") pod "dns-default-wxktg" (UID: "aafa76ab-df1f-4aba-9736-16236535b661") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:21.073345 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.073303    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aafa76ab-df1f-4aba-9736-16236535b661-tmp-dir\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:21.073487 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.073467    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aafa76ab-df1f-4aba-9736-16236535b661-config-volume\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:21.077192 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.077165    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5c5d52e-2d2c-429a-97e9-752108e3d730-cert\") pod \"ingress-canary-mvtdl\" (UID: \"e5c5d52e-2d2c-429a-97e9-752108e3d730\") " pod="openshift-ingress-canary/ingress-canary-mvtdl"
Jul 07 14:57:21.084863 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.084841    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv7qk\" (UniqueName: \"kubernetes.io/projected/e5c5d52e-2d2c-429a-97e9-752108e3d730-kube-api-access-cv7qk\") pod \"ingress-canary-mvtdl\" (UID: \"e5c5d52e-2d2c-429a-97e9-752108e3d730\") " pod="openshift-ingress-canary/ingress-canary-mvtdl"
Jul 07 14:57:21.085006 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.084987    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7dcp\" (UniqueName: \"kubernetes.io/projected/aafa76ab-df1f-4aba-9736-16236535b661-kube-api-access-l7dcp\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:21.131948 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.131902    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:21.234974 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.234870    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:21.239586 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.239569    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mvtdl"
Jul 07 14:57:21.336346 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.336052    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:21.384863 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.384830    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mvtdl"]
Jul 07 14:57:21.388482 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:21.388456    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c5d52e_2d2c_429a_97e9_752108e3d730.slice/crio-cbbda2818f7c2057483db74d3ea5a2d2b54485b12bdc5ccbd3899a3939409952 WatchSource:0}: Error finding container cbbda2818f7c2057483db74d3ea5a2d2b54485b12bdc5ccbd3899a3939409952: Status 404 returned error can't find the container with id cbbda2818f7c2057483db74d3ea5a2d2b54485b12bdc5ccbd3899a3939409952
Jul 07 14:57:21.511781 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.511642    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:21.511781 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.511642    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:21.511990 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.511644    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:21.514650 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.514628    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 14:57:21.514823 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.514806    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 14:57:21.514909 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.514852    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jrv9k\""
Jul 07 14:57:21.515860 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.515846    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 14:57:21.515939 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.515889    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 14:57:21.515939 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.515850    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8fcz7\""
Jul 07 14:57:21.577766 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.577734    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:21.577936 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:21.577888    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:21.577978 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:21.577958    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls podName:aafa76ab-df1f-4aba-9736-16236535b661 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:22.577941782 +0000 UTC m=+38.590338310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls") pod "dns-default-wxktg" (UID: "aafa76ab-df1f-4aba-9736-16236535b661") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:21.647622 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.647594    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:21.701104 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.701073    2569 generic.go:356] "Generic (PLEG): container finished" podID="b78593d3-7844-497b-96a9-17cd45cb49aa" containerID="e07fa07bf7669c272da532ce0e30e0bdddf5038a52aec71a00510556a6427245" exitCode=0
Jul 07 14:57:21.701297 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.701145    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2nvp9" event={"ID":"b78593d3-7844-497b-96a9-17cd45cb49aa","Type":"ContainerDied","Data":"e07fa07bf7669c272da532ce0e30e0bdddf5038a52aec71a00510556a6427245"}
Jul 07 14:57:21.702195 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.702178    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mvtdl" event={"ID":"e5c5d52e-2d2c-429a-97e9-752108e3d730","Type":"ContainerStarted","Data":"cbbda2818f7c2057483db74d3ea5a2d2b54485b12bdc5ccbd3899a3939409952"}
Jul 07 14:57:21.737839 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:21.737804    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:22.586809 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:22.586771    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:22.587394 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:22.586958    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:22.587394 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:22.587037    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls podName:aafa76ab-df1f-4aba-9736-16236535b661 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:24.587015812 +0000 UTC m=+40.599412342 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls") pod "dns-default-wxktg" (UID: "aafa76ab-df1f-4aba-9736-16236535b661") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:22.711124 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:22.711085    2569 generic.go:356] "Generic (PLEG): container finished" podID="b78593d3-7844-497b-96a9-17cd45cb49aa" containerID="3be156203c2c43da1c23f0d97f0d4cfe2b5e514f4f96e0050994376c7f000cb3" exitCode=0
Jul 07 14:57:22.711316 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:22.711139    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2nvp9" event={"ID":"b78593d3-7844-497b-96a9-17cd45cb49aa","Type":"ContainerDied","Data":"3be156203c2c43da1c23f0d97f0d4cfe2b5e514f4f96e0050994376c7f000cb3"}
Jul 07 14:57:23.074072 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.074033    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zwztf"]
Jul 07 14:57:23.078158 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.078130    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.082090 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.082067    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 14:57:23.083946 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.083924    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 14:57:23.084629 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.084607    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 14:57:23.084629 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.084618    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-85x7v\""
Jul 07 14:57:23.097163 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.097130    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 14:57:23.108597 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.108566    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zwztf"]
Jul 07 14:57:23.149202 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.149164    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:23.192200 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.192161    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.192200 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.192211    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d5b9610-2c56-40d7-a623-472a3224d65e-crio-socket\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.192479 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.192388    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.192479 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.192450    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksbhh\" (UniqueName: \"kubernetes.io/projected/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-api-access-ksbhh\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.192574 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.192484    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d5b9610-2c56-40d7-a623-472a3224d65e-data-volume\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.232998 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.232796    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:23.293645 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.293612    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.293645 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.293647    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d5b9610-2c56-40d7-a623-472a3224d65e-crio-socket\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.293869 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:23.293748    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:23.293869 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.293762    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.293869 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.293819    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksbhh\" (UniqueName: \"kubernetes.io/projected/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-api-access-ksbhh\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.293869 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:23.293838    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm podName:7d5b9610-2c56-40d7-a623-472a3224d65e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:23.793815572 +0000 UTC m=+39.806212104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zwztf" (UID: "7d5b9610-2c56-40d7-a623-472a3224d65e") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:23.294046 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.293879    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d5b9610-2c56-40d7-a623-472a3224d65e-data-volume\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.294046 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:23.293910    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:23.294046 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.293879    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d5b9610-2c56-40d7-a623-472a3224d65e-crio-socket\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.294046 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:23.293975    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls podName:7d5b9610-2c56-40d7-a623-472a3224d65e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:23.793958398 +0000 UTC m=+39.806354925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zwztf" (UID: "7d5b9610-2c56-40d7-a623-472a3224d65e") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:23.294496 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.294470    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d5b9610-2c56-40d7-a623-472a3224d65e-data-volume\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.307202 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.307182    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksbhh\" (UniqueName: \"kubernetes.io/projected/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-api-access-ksbhh\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.339437 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.339364    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:23.574162 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.574117    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:23.674294 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.674245    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:23.716886 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.716850    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2nvp9" event={"ID":"b78593d3-7844-497b-96a9-17cd45cb49aa","Type":"ContainerStarted","Data":"b6ff626d662acfe57ccf03b9a9ac0c44dbb2d2b0557e86d974d6cd41402419b8"}
Jul 07 14:57:23.742816 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.742767    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2nvp9" podStartSLOduration=9.224608035 podStartE2EDuration="38.742750212s" totalImagesPullingTime="29.518142177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:23.741661045 +0000 UTC m=+39.754057591" watchObservedRunningTime="2026-07-07 14:57:23.742750212 +0000 UTC m=+39.755146757"
Jul 07 14:57:23.797883 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.797847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.798032 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:23.797894    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:23.798032 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:23.797972    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:23.798032 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:23.797988    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:23.798155 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:23.798041    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls podName:7d5b9610-2c56-40d7-a623-472a3224d65e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:24.798027498 +0000 UTC m=+40.810424025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zwztf" (UID: "7d5b9610-2c56-40d7-a623-472a3224d65e") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:23.798155 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:23.798063    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm podName:7d5b9610-2c56-40d7-a623-472a3224d65e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:24.798049707 +0000 UTC m=+40.810446237 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zwztf" (UID: "7d5b9610-2c56-40d7-a623-472a3224d65e") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:24.061562 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.061522    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"]
Jul 07 14:57:24.074245 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.074216    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"
Jul 07 14:57:24.079168 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.079135    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 07 14:57:24.079538 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.079511    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 07 14:57:24.079656 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.079551    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8vnjc\""
Jul 07 14:57:24.079656 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.079588    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 07 14:57:24.079656 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.079509    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:57:24.080119 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.080101    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"]
Jul 07 14:57:24.083794 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.083771    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:24.177645 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.177608    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:24.200714 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.200623    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d15c4fa5-ac8a-4969-9ed6-bdf1fa681576-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-9dkqn\" (UID: \"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"
Jul 07 14:57:24.200714 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.200665    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d15c4fa5-ac8a-4969-9ed6-bdf1fa681576-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-9dkqn\" (UID: \"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"
Jul 07 14:57:24.200714 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.200687    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv9dn\" (UniqueName: \"kubernetes.io/projected/d15c4fa5-ac8a-4969-9ed6-bdf1fa681576-kube-api-access-qv9dn\") pod \"kube-storage-version-migrator-operator-846bf668f7-9dkqn\" (UID: \"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"
Jul 07 14:57:24.246655 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.246616    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:24.301476 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.301439    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d15c4fa5-ac8a-4969-9ed6-bdf1fa681576-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-9dkqn\" (UID: \"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"
Jul 07 14:57:24.301638 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.301484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d15c4fa5-ac8a-4969-9ed6-bdf1fa681576-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-9dkqn\" (UID: \"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"
Jul 07 14:57:24.301638 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.301607    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv9dn\" (UniqueName: \"kubernetes.io/projected/d15c4fa5-ac8a-4969-9ed6-bdf1fa681576-kube-api-access-qv9dn\") pod \"kube-storage-version-migrator-operator-846bf668f7-9dkqn\" (UID: \"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"
Jul 07 14:57:24.302045 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.302024    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d15c4fa5-ac8a-4969-9ed6-bdf1fa681576-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-9dkqn\" (UID: \"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"
Jul 07 14:57:24.303938 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.303920    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d15c4fa5-ac8a-4969-9ed6-bdf1fa681576-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-9dkqn\" (UID: \"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"
Jul 07 14:57:24.311157 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.311140    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv9dn\" (UniqueName: \"kubernetes.io/projected/d15c4fa5-ac8a-4969-9ed6-bdf1fa681576-kube-api-access-qv9dn\") pod \"kube-storage-version-migrator-operator-846bf668f7-9dkqn\" (UID: \"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"
Jul 07 14:57:24.385298 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.385238    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"
Jul 07 14:57:24.545784 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.545752    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn"]
Jul 07 14:57:24.548627 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:24.548603    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd15c4fa5_ac8a_4969_9ed6_bdf1fa681576.slice/crio-ad5218dc15db27798a4176a13f6782994472cfdcfec782222f254c0fc7a9cb12 WatchSource:0}: Error finding container ad5218dc15db27798a4176a13f6782994472cfdcfec782222f254c0fc7a9cb12: Status 404 returned error can't find the container with id ad5218dc15db27798a4176a13f6782994472cfdcfec782222f254c0fc7a9cb12
Jul 07 14:57:24.587895 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.587872    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:24.604788 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.604755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:24.604922 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:24.604906    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:24.604977 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:24.604969    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls podName:aafa76ab-df1f-4aba-9736-16236535b661 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:28.604953335 +0000 UTC m=+44.617349859 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls") pod "dns-default-wxktg" (UID: "aafa76ab-df1f-4aba-9736-16236535b661") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:24.658182 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.658157    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cmwns_03e07f30-8257-4a4b-bb01-042bb672a64c/dns-node-resolver/0.log"
Jul 07 14:57:24.704664 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.704635    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:24.720702 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.720637    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mvtdl" event={"ID":"e5c5d52e-2d2c-429a-97e9-752108e3d730","Type":"ContainerStarted","Data":"1247e889e21e416067943a210c6b8ae7417cf0e6a38b29f2ef4e061f073ac984"}
Jul 07 14:57:24.721818 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.721798    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn" event={"ID":"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576","Type":"ContainerStarted","Data":"ad5218dc15db27798a4176a13f6782994472cfdcfec782222f254c0fc7a9cb12"}
Jul 07 14:57:24.769382 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.769337    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mvtdl" podStartSLOduration=2.239686673 podStartE2EDuration="4.769321023s" totalImagesPullingTime="2.52963435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:24.767867075 +0000 UTC m=+40.780263621" watchObservedRunningTime="2026-07-07 14:57:24.769321023 +0000 UTC m=+40.781717590"
Jul 07 14:57:24.805964 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.805923    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:24.806105 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:24.806013    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:24.806105 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:24.806039    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:24.806217 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:24.806109    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm podName:7d5b9610-2c56-40d7-a623-472a3224d65e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:26.806089527 +0000 UTC m=+42.818486051 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zwztf" (UID: "7d5b9610-2c56-40d7-a623-472a3224d65e") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:24.806217 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:24.806165    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:24.806311 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:24.806225    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls podName:7d5b9610-2c56-40d7-a623-472a3224d65e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:26.806209396 +0000 UTC m=+42.818605929 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zwztf" (UID: "7d5b9610-2c56-40d7-a623-472a3224d65e") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:25.455288 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:25.455242    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ftfzh_6aaa686c-76ef-4753-86e1-5df48442b8ce/node-ca/0.log"
Jul 07 14:57:26.167064 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.167034    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"]
Jul 07 14:57:26.189047 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.189015    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"
Jul 07 14:57:26.197162 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.197133    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 07 14:57:26.197464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.197434    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-k7dm5\""
Jul 07 14:57:26.197464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.197446    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 07 14:57:26.197464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.197438    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:57:26.198762 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.198700    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 07 14:57:26.199585 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.199567    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"]
Jul 07 14:57:26.256214 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.256148    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mvtdl_e5c5d52e-2d2c-429a-97e9-752108e3d730/serve-healthcheck-canary/0.log"
Jul 07 14:57:26.318536 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.318501    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzx9q\" (UniqueName: \"kubernetes.io/projected/ae5fdbe6-b912-4300-a751-d513902c68fa-kube-api-access-qzx9q\") pod \"service-ca-operator-5d46d9ffd4-wk7pm\" (UID: \"ae5fdbe6-b912-4300-a751-d513902c68fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"
Jul 07 14:57:26.318700 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.318591    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ae5fdbe6-b912-4300-a751-d513902c68fa-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-wk7pm\" (UID: \"ae5fdbe6-b912-4300-a751-d513902c68fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"
Jul 07 14:57:26.318700 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.318624    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ae5fdbe6-b912-4300-a751-d513902c68fa-config\") pod \"service-ca-operator-5d46d9ffd4-wk7pm\" (UID: \"ae5fdbe6-b912-4300-a751-d513902c68fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"
Jul 07 14:57:26.419194 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.419130    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzx9q\" (UniqueName: \"kubernetes.io/projected/ae5fdbe6-b912-4300-a751-d513902c68fa-kube-api-access-qzx9q\") pod \"service-ca-operator-5d46d9ffd4-wk7pm\" (UID: \"ae5fdbe6-b912-4300-a751-d513902c68fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"
Jul 07 14:57:26.419362 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.419232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ae5fdbe6-b912-4300-a751-d513902c68fa-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-wk7pm\" (UID: \"ae5fdbe6-b912-4300-a751-d513902c68fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"
Jul 07 14:57:26.419362 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.419292    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ae5fdbe6-b912-4300-a751-d513902c68fa-config\") pod \"service-ca-operator-5d46d9ffd4-wk7pm\" (UID: \"ae5fdbe6-b912-4300-a751-d513902c68fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"
Jul 07 14:57:26.419855 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.419831    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ae5fdbe6-b912-4300-a751-d513902c68fa-config\") pod \"service-ca-operator-5d46d9ffd4-wk7pm\" (UID: \"ae5fdbe6-b912-4300-a751-d513902c68fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"
Jul 07 14:57:26.421593 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.421566    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ae5fdbe6-b912-4300-a751-d513902c68fa-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-wk7pm\" (UID: \"ae5fdbe6-b912-4300-a751-d513902c68fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"
Jul 07 14:57:26.428394 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.428371    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzx9q\" (UniqueName: \"kubernetes.io/projected/ae5fdbe6-b912-4300-a751-d513902c68fa-kube-api-access-qzx9q\") pod \"service-ca-operator-5d46d9ffd4-wk7pm\" (UID: \"ae5fdbe6-b912-4300-a751-d513902c68fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"
Jul 07 14:57:26.497545 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.497519    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"
Jul 07 14:57:26.823367 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.823276    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:26.823367 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:26.823336    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:26.823587 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:26.823414    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:26.823587 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:26.823447    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:26.823587 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:26.823487    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm podName:7d5b9610-2c56-40d7-a623-472a3224d65e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:30.823466092 +0000 UTC m=+46.835862616 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zwztf" (UID: "7d5b9610-2c56-40d7-a623-472a3224d65e") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:26.823587 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:26.823524    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls podName:7d5b9610-2c56-40d7-a623-472a3224d65e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:30.823509042 +0000 UTC m=+46.835905569 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zwztf" (UID: "7d5b9610-2c56-40d7-a623-472a3224d65e") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:27.012833 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.012802    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm"]
Jul 07 14:57:27.015969 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:27.015941    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae5fdbe6_b912_4300_a751_d513902c68fa.slice/crio-4d1b9c8a82f4479a3408d137ff0beec8fa4d81a0b6b3763083c48f4d804b7f4f WatchSource:0}: Error finding container 4d1b9c8a82f4479a3408d137ff0beec8fa4d81a0b6b3763083c48f4d804b7f4f: Status 404 returned error can't find the container with id 4d1b9c8a82f4479a3408d137ff0beec8fa4d81a0b6b3763083c48f4d804b7f4f
Jul 07 14:57:27.086994 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.086927    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-4jf9k"]
Jul 07 14:57:27.105116 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.105091    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-4jf9k"]
Jul 07 14:57:27.105246 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.105194    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4jf9k"
Jul 07 14:57:27.107822 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.107805    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qws79\""
Jul 07 14:57:27.125216 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.125182    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcqqp\" (UniqueName: \"kubernetes.io/projected/9bdcf86f-3d19-4170-b895-e30423ad9812-kube-api-access-wcqqp\") pod \"network-check-source-989f8b8dc-4jf9k\" (UID: \"9bdcf86f-3d19-4170-b895-e30423ad9812\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4jf9k"
Jul 07 14:57:27.226247 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.226214    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcqqp\" (UniqueName: \"kubernetes.io/projected/9bdcf86f-3d19-4170-b895-e30423ad9812-kube-api-access-wcqqp\") pod \"network-check-source-989f8b8dc-4jf9k\" (UID: \"9bdcf86f-3d19-4170-b895-e30423ad9812\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4jf9k"
Jul 07 14:57:27.235627 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.235602    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcqqp\" (UniqueName: \"kubernetes.io/projected/9bdcf86f-3d19-4170-b895-e30423ad9812-kube-api-access-wcqqp\") pod \"network-check-source-989f8b8dc-4jf9k\" (UID: \"9bdcf86f-3d19-4170-b895-e30423ad9812\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4jf9k"
Jul 07 14:57:27.414351 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.414329    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4jf9k"
Jul 07 14:57:27.550710 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.550678    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6874fb6b8b-67h7c"]
Jul 07 14:57:27.580629 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.580606    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6874fb6b8b-67h7c"]
Jul 07 14:57:27.580629 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.580640    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-4jf9k"]
Jul 07 14:57:27.580799 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.580743    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.583302 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:27.583276    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bdcf86f_3d19_4170_b895_e30423ad9812.slice/crio-3ef469e4ed0c0a934102b0a351c35556b47d46e24b4e9e27ddbced85dcdc42ee WatchSource:0}: Error finding container 3ef469e4ed0c0a934102b0a351c35556b47d46e24b4e9e27ddbced85dcdc42ee: Status 404 returned error can't find the container with id 3ef469e4ed0c0a934102b0a351c35556b47d46e24b4e9e27ddbced85dcdc42ee
Jul 07 14:57:27.585659 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.585475    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 14:57:27.585905 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.585751    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 14:57:27.585905 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.585754    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 14:57:27.588676 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.587062    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mrpzk\""
Jul 07 14:57:27.591044 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.591023    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 14:57:27.629339 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.629306    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.629515 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.629358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-bound-sa-token\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.629515 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.629377    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7fkr\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-kube-api-access-x7fkr\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.629515 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.629485    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-ca-trust-extracted\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.629643 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.629535    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-trusted-ca\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.629643 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.629597    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-image-registry-private-configuration\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.629643 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.629622    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-certificates\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.629808 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.629647    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-installation-pull-secrets\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.730427 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.730391    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-ca-trust-extracted\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.730594 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.730463    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-trusted-ca\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.730594 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.730533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-image-registry-private-configuration\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.730594 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.730568    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-certificates\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.730739 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.730599    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-installation-pull-secrets\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.730739 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.730646    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.730739 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.730700    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-bound-sa-token\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.730739 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.730729    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7fkr\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-kube-api-access-x7fkr\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.731573 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:27.731429    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:27.731573 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:27.731452    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6874fb6b8b-67h7c: secret "image-registry-tls" not found
Jul 07 14:57:27.731573 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:27.731519    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls podName:f70009d8-1c75-4b71-8836-ee0f6ec71ca9 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:28.231500179 +0000 UTC m=+44.243896720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls") pod "image-registry-6874fb6b8b-67h7c" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9") : secret "image-registry-tls" not found
Jul 07 14:57:27.731573 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.731506    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-ca-trust-extracted\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.734294 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.734266    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-certificates\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.734961 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.734920    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-installation-pull-secrets\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.735758 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.735732    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-trusted-ca\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.736771 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.736584    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn" event={"ID":"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576","Type":"ContainerStarted","Data":"fdf747efd00637ee94e4e4188928ca9d7e8da121d528d51d26a10f1121ac112e"}
Jul 07 14:57:27.736771 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.736722    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-image-registry-private-configuration\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.738418 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.738357    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm" event={"ID":"ae5fdbe6-b912-4300-a751-d513902c68fa","Type":"ContainerStarted","Data":"4d1b9c8a82f4479a3408d137ff0beec8fa4d81a0b6b3763083c48f4d804b7f4f"}
Jul 07 14:57:27.740140 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.740117    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-bound-sa-token\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.740673 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.740628    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4jf9k" event={"ID":"9bdcf86f-3d19-4170-b895-e30423ad9812","Type":"ContainerStarted","Data":"3ef469e4ed0c0a934102b0a351c35556b47d46e24b4e9e27ddbced85dcdc42ee"}
Jul 07 14:57:27.740776 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.740754    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7fkr\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-kube-api-access-x7fkr\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:27.756005 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:27.755954    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn" podStartSLOduration=1.011001836 podStartE2EDuration="3.755939934s" totalImagesPullingTime="2.744938098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:27.75469334 +0000 UTC m=+43.767089887" watchObservedRunningTime="2026-07-07 14:57:27.755939934 +0000 UTC m=+43.768336505"
Jul 07 14:57:28.235400 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:28.235357    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:28.235852 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:28.235495    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:28.235852 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:28.235515    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6874fb6b8b-67h7c: secret "image-registry-tls" not found
Jul 07 14:57:28.235852 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:28.235582    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls podName:f70009d8-1c75-4b71-8836-ee0f6ec71ca9 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:29.235563485 +0000 UTC m=+45.247960020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls") pod "image-registry-6874fb6b8b-67h7c" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9") : secret "image-registry-tls" not found
Jul 07 14:57:28.638280 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:28.638107    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:28.638434 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:28.638380    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:28.638543 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:28.638444    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls podName:aafa76ab-df1f-4aba-9736-16236535b661 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:36.638426203 +0000 UTC m=+52.650822727 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls") pod "dns-default-wxktg" (UID: "aafa76ab-df1f-4aba-9736-16236535b661") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:29.243062 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:29.243019    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:29.243680 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:29.243191    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:29.243680 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:29.243214    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6874fb6b8b-67h7c: secret "image-registry-tls" not found
Jul 07 14:57:29.243680 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:29.243299    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls podName:f70009d8-1c75-4b71-8836-ee0f6ec71ca9 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:31.243279062 +0000 UTC m=+47.255675588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls") pod "image-registry-6874fb6b8b-67h7c" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9") : secret "image-registry-tls" not found
Jul 07 14:57:29.480244 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:29.480196    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:29.573926 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:29.573825    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:29.748400 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:29.748321    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm" event={"ID":"ae5fdbe6-b912-4300-a751-d513902c68fa","Type":"ContainerStarted","Data":"9b2d0fcc23a90848346b9a167b87bd769ffc4fc6652559007af516d5fe800752"}
Jul 07 14:57:29.768204 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:29.768146    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm" podStartSLOduration=1.492770067 podStartE2EDuration="3.768126514s" totalImagesPullingTime="2.275356447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:29.767408254 +0000 UTC m=+45.779804849" watchObservedRunningTime="2026-07-07 14:57:29.768126514 +0000 UTC m=+45.780523063"
Jul 07 14:57:30.471282 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.471237    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-f8tqp"]
Jul 07 14:57:30.504877 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.504843    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-f8tqp"]
Jul 07 14:57:30.505065 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.504858    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f8tqp"
Jul 07 14:57:30.508219 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.508193    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 07 14:57:30.508378 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.508193    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-v47vm\""
Jul 07 14:57:30.509157 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.509108    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 07 14:57:30.517900 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.517877    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:30.553984 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.553951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44vtt\" (UniqueName: \"kubernetes.io/projected/8ae9950b-2803-432d-946f-6b22adb23e62-kube-api-access-44vtt\") pod \"migrator-d57d58cc9-f8tqp\" (UID: \"8ae9950b-2803-432d-946f-6b22adb23e62\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f8tqp"
Jul 07 14:57:30.581822 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.581788    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:30.654858 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.654823    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44vtt\" (UniqueName: \"kubernetes.io/projected/8ae9950b-2803-432d-946f-6b22adb23e62-kube-api-access-44vtt\") pod \"migrator-d57d58cc9-f8tqp\" (UID: \"8ae9950b-2803-432d-946f-6b22adb23e62\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f8tqp"
Jul 07 14:57:30.670785 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.670757    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44vtt\" (UniqueName: \"kubernetes.io/projected/8ae9950b-2803-432d-946f-6b22adb23e62-kube-api-access-44vtt\") pod \"migrator-d57d58cc9-f8tqp\" (UID: \"8ae9950b-2803-432d-946f-6b22adb23e62\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f8tqp"
Jul 07 14:57:30.816222 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.816142    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f8tqp"
Jul 07 14:57:30.856470 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.856431    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:30.856665 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:30.856586    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:57:30.856736 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:30.856679    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm podName:7d5b9610-2c56-40d7-a623-472a3224d65e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:38.856658065 +0000 UTC m=+54.869054618 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zwztf" (UID: "7d5b9610-2c56-40d7-a623-472a3224d65e") : configmap "kube-rbac-proxy" not found
Jul 07 14:57:30.856801 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:30.856750    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:30.857043 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:30.857024    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:30.857134 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:30.857073    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls podName:7d5b9610-2c56-40d7-a623-472a3224d65e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:38.85705853 +0000 UTC m=+54.869455054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zwztf" (UID: "7d5b9610-2c56-40d7-a623-472a3224d65e") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:31.169454 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:31.169422    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-f8tqp"]
Jul 07 14:57:31.172294 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:31.172270    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ae9950b_2803_432d_946f_6b22adb23e62.slice/crio-7b00a3feb92ff483fa19b3cad46e4c10f280f1ad0ac7e25c1098fa6e907083d0 WatchSource:0}: Error finding container 7b00a3feb92ff483fa19b3cad46e4c10f280f1ad0ac7e25c1098fa6e907083d0: Status 404 returned error can't find the container with id 7b00a3feb92ff483fa19b3cad46e4c10f280f1ad0ac7e25c1098fa6e907083d0
Jul 07 14:57:31.260379 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:31.260348    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:31.260532 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:31.260472    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:31.260532 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:31.260485    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6874fb6b8b-67h7c: secret "image-registry-tls" not found
Jul 07 14:57:31.260628 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:31.260534    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls podName:f70009d8-1c75-4b71-8836-ee0f6ec71ca9 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:35.260520348 +0000 UTC m=+51.272916872 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls") pod "image-registry-6874fb6b8b-67h7c" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9") : secret "image-registry-tls" not found
Jul 07 14:57:31.669009 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:31.668980    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:31.755732 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:31.755692    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4jf9k" event={"ID":"9bdcf86f-3d19-4170-b895-e30423ad9812","Type":"ContainerStarted","Data":"15bff08c0b52284343449ccab3677456553ae1cf650ae5346c0cc2db60e48799"}
Jul 07 14:57:31.757146 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:31.757122    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f8tqp" event={"ID":"8ae9950b-2803-432d-946f-6b22adb23e62","Type":"ContainerStarted","Data":"7b00a3feb92ff483fa19b3cad46e4c10f280f1ad0ac7e25c1098fa6e907083d0"}
Jul 07 14:57:31.770645 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:31.770603    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:31.773736 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:31.773688    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4jf9k" podStartSLOduration=0.964980115 podStartE2EDuration="4.773669194s" totalImagesPullingTime="3.808689079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:31.772644297 +0000 UTC m=+47.785040870" watchObservedRunningTime="2026-07-07 14:57:31.773669194 +0000 UTC m=+47.786065740"
Jul 07 14:57:31.914997 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:31.914958    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:32.015873 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:32.015786    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:32.368957 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:32.368858    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:32.469994 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:32.469964    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:33.766926 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:33.766886    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f8tqp" event={"ID":"8ae9950b-2803-432d-946f-6b22adb23e62","Type":"ContainerStarted","Data":"af0125341297bd7100043aecd2465b718b3bf70192139db30b0cc0c0aeeb15af"}
Jul 07 14:57:33.767346 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:33.766948    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f8tqp" event={"ID":"8ae9950b-2803-432d-946f-6b22adb23e62","Type":"ContainerStarted","Data":"73f84a2496a7b50f95a1404256d296347ac8c196cd3f39f9b86bee850a9a5512"}
Jul 07 14:57:33.790863 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:33.790816    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-f8tqp" podStartSLOduration=2.085280082 podStartE2EDuration="3.79080031s" totalImagesPullingTime="1.705520228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:33.789378245 +0000 UTC m=+49.801774792" watchObservedRunningTime="2026-07-07 14:57:33.79080031 +0000 UTC m=+49.803196855"
Jul 07 14:57:33.868466 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:33.868427    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:33.973235 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:33.973198    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:34.155750 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:34.155663    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:34.267652 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:34.267616    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:35.297046 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:35.297003    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:35.297476 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:35.297158    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:57:35.297476 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:35.297180    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6874fb6b8b-67h7c: secret "image-registry-tls" not found
Jul 07 14:57:35.297476 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:35.297235    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls podName:f70009d8-1c75-4b71-8836-ee0f6ec71ca9 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:43.297220295 +0000 UTC m=+59.309616818 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls") pod "image-registry-6874fb6b8b-67h7c" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9") : secret "image-registry-tls" not found
Jul 07 14:57:35.310604 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:35.310564    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:36.254676 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:36.254642    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:36.710660 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:36.710625    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:36.711089 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:36.710790    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:57:36.711089 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:36.710861    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls podName:aafa76ab-df1f-4aba-9736-16236535b661 nodeName:}" failed. No retries permitted until 2026-07-07 14:57:52.710843257 +0000 UTC m=+68.723239780 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls") pod "dns-default-wxktg" (UID: "aafa76ab-df1f-4aba-9736-16236535b661") : secret "dns-default-metrics-tls" not found
Jul 07 14:57:37.711567 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:37.711534    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:38.368784 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:38.368752    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:38.929688 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:38.929658    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:38.930097 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:38.929705    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:38.930097 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:38.929819    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:38.930097 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:57:38.929881    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls podName:7d5b9610-2c56-40d7-a623-472a3224d65e nodeName:}" failed. No retries permitted until 2026-07-07 14:57:54.929866722 +0000 UTC m=+70.942263247 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zwztf" (UID: "7d5b9610-2c56-40d7-a623-472a3224d65e") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:57:38.930221 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:38.930133    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d5b9610-2c56-40d7-a623-472a3224d65e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:38.996287 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:38.996244    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:39.094331 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:39.094294    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:39.200981 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:39.200901    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:39.322085 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:39.322054    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:39.905380 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:39.905343    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:40.473859 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:40.473827    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:40.933590 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:40.933559    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:41.074934 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:41.074899    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:41.375449 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:41.375357    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:41.488223 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:41.488194    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:42.286480 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:42.286450    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:42.569688 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:42.569603    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:42.829759 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:42.829670    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:42.950602 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:42.950569    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:43.072974 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:43.072949    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:43.283978 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:43.283946    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:43.367381 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:43.367332    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:43.369648 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:43.369620    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls\") pod \"image-registry-6874fb6b8b-67h7c\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") " pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:43.377295 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:43.377272    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:43.497991 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:43.497955    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:43.627114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:43.627081    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6874fb6b8b-67h7c"]
Jul 07 14:57:43.630117 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:43.630089    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf70009d8_1c75_4b71_8836_ee0f6ec71ca9.slice/crio-c0775fa0bdc5111d29bde862cff120fd25a4b2a060edf1ce107439275244f2f2 WatchSource:0}: Error finding container c0775fa0bdc5111d29bde862cff120fd25a4b2a060edf1ce107439275244f2f2: Status 404 returned error can't find the container with id c0775fa0bdc5111d29bde862cff120fd25a4b2a060edf1ce107439275244f2f2
Jul 07 14:57:43.804045 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:43.803960    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c" event={"ID":"f70009d8-1c75-4b71-8836-ee0f6ec71ca9","Type":"ContainerStarted","Data":"f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5"}
Jul 07 14:57:43.804045 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:43.804014    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c" event={"ID":"f70009d8-1c75-4b71-8836-ee0f6ec71ca9","Type":"ContainerStarted","Data":"c0775fa0bdc5111d29bde862cff120fd25a4b2a060edf1ce107439275244f2f2"}
Jul 07 14:57:43.804228 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:43.804083    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:57:43.829732 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:43.829682    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c" podStartSLOduration=16.829666956 podStartE2EDuration="16.829666956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:43.829242074 +0000 UTC m=+59.841638622" watchObservedRunningTime="2026-07-07 14:57:43.829666956 +0000 UTC m=+59.842063518"
Jul 07 14:57:43.927666 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:43.927633    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:44.044623 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:44.044588    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:44.228093 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:44.228063    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:44.467102 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:44.467067    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:44.611714 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:44.611621    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:44.711952 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:44.711919    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:44.878521 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:44.878425    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:45.009497 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:45.009458    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:45.112453 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:45.112417    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:45.288656 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:45.288619    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:45.363680 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:45.363646    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:45.443695 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:45.443666    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:45.698082 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:45.698056    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gpxk2"
Jul 07 14:57:45.818540 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:45.818508    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:46.285978 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:46.285931    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:46.647626 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:46.647542    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:46.736947 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:46.736912    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:46.876129 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:46.876095    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:47.081514 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:47.081476    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:47.470798 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:47.470766    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:48.073611 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:48.073575    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:48.476604 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:48.476562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:48.675220 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:48.675186    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:49.290463 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:49.290432    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:49.670470 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:49.670439    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:49.871678 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:49.871648    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:50.080908 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.080816    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:50.225775 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.225738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:50.228531 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.228511    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 14:57:50.238930 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.238905    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f99558a-08a5-4532-a131-eb30f2278bb5-original-pull-secret\") pod \"global-pull-secret-syncer-4rqtz\" (UID: \"5f99558a-08a5-4532-a131-eb30f2278bb5\") " pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:50.326307 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.326230    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wwxg\" (UniqueName: \"kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg\") pod \"network-check-target-5w2jq\" (UID: \"213be297-4d57-4440-9531-e4121fe87b19\") " pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:50.326711 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.326328    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:50.328743 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.328720    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wwxg\" (UniqueName: \"kubernetes.io/projected/213be297-4d57-4440-9531-e4121fe87b19-kube-api-access-7wwxg\") pod \"network-check-target-5w2jq\" (UID: \"213be297-4d57-4440-9531-e4121fe87b19\") " pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:50.329345 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.329328    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 14:57:50.332985 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.332933    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4rqtz"
Jul 07 14:57:50.339161 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.339132    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fcdb256-eea5-428e-9492-2cd608fd8a23-metrics-certs\") pod \"network-metrics-daemon-s5zzk\" (UID: \"7fcdb256-eea5-428e-9492-2cd608fd8a23\") " pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:50.462449 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.462419    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4rqtz"]
Jul 07 14:57:50.465189 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:50.465155    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f99558a_08a5_4532_a131_eb30f2278bb5.slice/crio-aa2af1248d866d8580ee78687a2925ddc565bbdf4c191785c7938166bcbd508f WatchSource:0}: Error finding container aa2af1248d866d8580ee78687a2925ddc565bbdf4c191785c7938166bcbd508f: Status 404 returned error can't find the container with id aa2af1248d866d8580ee78687a2925ddc565bbdf4c191785c7938166bcbd508f
Jul 07 14:57:50.625343 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.625239    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8fcz7\""
Jul 07 14:57:50.632154 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.632133    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jrv9k\""
Jul 07 14:57:50.633111 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.633093    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5zzk"
Jul 07 14:57:50.639714 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.639693    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:50.781729 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.781702    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s5zzk"]
Jul 07 14:57:50.782375 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:50.782349    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fcdb256_eea5_428e_9492_2cd608fd8a23.slice/crio-9ef448a7bf812847749e68395e2758feb35360f574de702dd7a269af7eec9197 WatchSource:0}: Error finding container 9ef448a7bf812847749e68395e2758feb35360f574de702dd7a269af7eec9197: Status 404 returned error can't find the container with id 9ef448a7bf812847749e68395e2758feb35360f574de702dd7a269af7eec9197
Jul 07 14:57:50.786536 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.786514    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5w2jq"]
Jul 07 14:57:50.789168 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:50.789147    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod213be297_4d57_4440_9531_e4121fe87b19.slice/crio-ad52ae926301917d51d439a8b3c221d23865384dae9b9628b3a92913e37509a6 WatchSource:0}: Error finding container ad52ae926301917d51d439a8b3c221d23865384dae9b9628b3a92913e37509a6: Status 404 returned error can't find the container with id ad52ae926301917d51d439a8b3c221d23865384dae9b9628b3a92913e37509a6
Jul 07 14:57:50.828310 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.828283    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s5zzk" event={"ID":"7fcdb256-eea5-428e-9492-2cd608fd8a23","Type":"ContainerStarted","Data":"9ef448a7bf812847749e68395e2758feb35360f574de702dd7a269af7eec9197"}
Jul 07 14:57:50.829284 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.829261    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4rqtz" event={"ID":"5f99558a-08a5-4532-a131-eb30f2278bb5","Type":"ContainerStarted","Data":"aa2af1248d866d8580ee78687a2925ddc565bbdf4c191785c7938166bcbd508f"}
Jul 07 14:57:50.830726 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:50.830705    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5w2jq" event={"ID":"213be297-4d57-4440-9531-e4121fe87b19","Type":"ContainerStarted","Data":"ad52ae926301917d51d439a8b3c221d23865384dae9b9628b3a92913e37509a6"}
Jul 07 14:57:51.836688 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:51.836647    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5w2jq" event={"ID":"213be297-4d57-4440-9531-e4121fe87b19","Type":"ContainerStarted","Data":"4a38ccdf1041e729a7680bdbe4cc1f194a5af7bd4edc8bd65498958aa0eb5a95"}
Jul 07 14:57:51.837289 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:51.837244    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:57:52.745816 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:52.745784    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:52.748445 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:52.748412    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aafa76ab-df1f-4aba-9736-16236535b661-metrics-tls\") pod \"dns-default-wxktg\" (UID: \"aafa76ab-df1f-4aba-9736-16236535b661\") " pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:52.754015 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:52.753974    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:52.842094 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:52.842058    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s5zzk" event={"ID":"7fcdb256-eea5-428e-9492-2cd608fd8a23","Type":"ContainerStarted","Data":"bc3eac0c800bf60e9730c5dbab64d49faac96b3803383103b3f395572dfd80aa"}
Jul 07 14:57:52.842618 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:52.842118    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s5zzk" event={"ID":"7fcdb256-eea5-428e-9492-2cd608fd8a23","Type":"ContainerStarted","Data":"d9fb7d5cf45d469b4d9403ec70bc33bf4f146fc74e7f4bc79c1d002ff9729459"}
Jul 07 14:57:52.858478 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:52.858441    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:52.864502 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:52.864403    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s5zzk" podStartSLOduration=66.844356923 podStartE2EDuration="1m7.864381841s" totalImagesPullingTime="1.020024918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:52.863488878 +0000 UTC m=+68.875885450" watchObservedRunningTime="2026-07-07 14:57:52.864381841 +0000 UTC m=+68.876778389"
Jul 07 14:57:52.865158 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:52.865126    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5w2jq" podStartSLOduration=67.865114181 podStartE2EDuration="1m7.865114181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:51.854426412 +0000 UTC m=+67.866822959" watchObservedRunningTime="2026-07-07 14:57:52.865114181 +0000 UTC m=+68.877510729"
Jul 07 14:57:52.953836 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:52.953775    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:53.037046 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.036973    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pkm7x\""
Jul 07 14:57:53.045195 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.045171    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wxktg"
Jul 07 14:57:53.137609 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.137558    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:53.195388 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.195353    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wxktg"]
Jul 07 14:57:53.720594 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.720559    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"]
Jul 07 14:57:53.760388 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.752616    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf"]
Jul 07 14:57:53.760388 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.753755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:57:53.766277 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.764407    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 14:57:53.766277 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.765890    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 14:57:53.766277 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.766101    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 14:57:53.766595 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.766489    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"]
Jul 07 14:57:53.766595 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.766540    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf"]
Jul 07 14:57:53.766708 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.766627    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf"
Jul 07 14:57:53.770891 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.770712    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 07 14:57:53.771294 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.771272    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-qpdt6\""
Jul 07 14:57:53.771528 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.771408    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 07 14:57:53.810220 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.810185    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6874fb6b8b-67h7c"]
Jul 07 14:57:53.864943 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.864911    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc74be8d-e68b-416c-bf7c-231a70cd023e-tmp\") pod \"klusterlet-addon-workmgr-7646fd4cb5-gdrks\" (UID: \"bc74be8d-e68b-416c-bf7c-231a70cd023e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:57:53.865380 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.864964    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bc74be8d-e68b-416c-bf7c-231a70cd023e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7646fd4cb5-gdrks\" (UID: \"bc74be8d-e68b-416c-bf7c-231a70cd023e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:57:53.865380 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.864994    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jxs2\" (UniqueName: \"kubernetes.io/projected/a10b510f-c07f-41d8-9f59-315b51a817fd-kube-api-access-2jxs2\") pod \"managed-serviceaccount-addon-agent-6867bb98d-tz9vf\" (UID: \"a10b510f-c07f-41d8-9f59-315b51a817fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf"
Jul 07 14:57:53.865380 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.865053    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a10b510f-c07f-41d8-9f59-315b51a817fd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6867bb98d-tz9vf\" (UID: \"a10b510f-c07f-41d8-9f59-315b51a817fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf"
Jul 07 14:57:53.865380 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.865086    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmdks\" (UniqueName: \"kubernetes.io/projected/bc74be8d-e68b-416c-bf7c-231a70cd023e-kube-api-access-pmdks\") pod \"klusterlet-addon-workmgr-7646fd4cb5-gdrks\" (UID: \"bc74be8d-e68b-416c-bf7c-231a70cd023e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:57:53.966276 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.966210    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc74be8d-e68b-416c-bf7c-231a70cd023e-tmp\") pod \"klusterlet-addon-workmgr-7646fd4cb5-gdrks\" (UID: \"bc74be8d-e68b-416c-bf7c-231a70cd023e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:57:53.966468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.966343    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bc74be8d-e68b-416c-bf7c-231a70cd023e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7646fd4cb5-gdrks\" (UID: \"bc74be8d-e68b-416c-bf7c-231a70cd023e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:57:53.966468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.966397    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxs2\" (UniqueName: \"kubernetes.io/projected/a10b510f-c07f-41d8-9f59-315b51a817fd-kube-api-access-2jxs2\") pod \"managed-serviceaccount-addon-agent-6867bb98d-tz9vf\" (UID: \"a10b510f-c07f-41d8-9f59-315b51a817fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf"
Jul 07 14:57:53.966468 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.966434    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a10b510f-c07f-41d8-9f59-315b51a817fd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6867bb98d-tz9vf\" (UID: \"a10b510f-c07f-41d8-9f59-315b51a817fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf"
Jul 07 14:57:53.966721 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.966482    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmdks\" (UniqueName: \"kubernetes.io/projected/bc74be8d-e68b-416c-bf7c-231a70cd023e-kube-api-access-pmdks\") pod \"klusterlet-addon-workmgr-7646fd4cb5-gdrks\" (UID: \"bc74be8d-e68b-416c-bf7c-231a70cd023e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:57:53.966721 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.966625    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc74be8d-e68b-416c-bf7c-231a70cd023e-tmp\") pod \"klusterlet-addon-workmgr-7646fd4cb5-gdrks\" (UID: \"bc74be8d-e68b-416c-bf7c-231a70cd023e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:57:53.969710 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.969687    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a10b510f-c07f-41d8-9f59-315b51a817fd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6867bb98d-tz9vf\" (UID: \"a10b510f-c07f-41d8-9f59-315b51a817fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf"
Jul 07 14:57:53.969710 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.969699    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bc74be8d-e68b-416c-bf7c-231a70cd023e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7646fd4cb5-gdrks\" (UID: \"bc74be8d-e68b-416c-bf7c-231a70cd023e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:57:53.990416 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.990345    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmdks\" (UniqueName: \"kubernetes.io/projected/bc74be8d-e68b-416c-bf7c-231a70cd023e-kube-api-access-pmdks\") pod \"klusterlet-addon-workmgr-7646fd4cb5-gdrks\" (UID: \"bc74be8d-e68b-416c-bf7c-231a70cd023e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:57:53.992376 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.992342    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jxs2\" (UniqueName: \"kubernetes.io/projected/a10b510f-c07f-41d8-9f59-315b51a817fd-kube-api-access-2jxs2\") pod \"managed-serviceaccount-addon-agent-6867bb98d-tz9vf\" (UID: \"a10b510f-c07f-41d8-9f59-315b51a817fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf"
Jul 07 14:57:53.994575 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:53.994497    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:54.072595 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:54.072564    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:57:54.087761 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:54.087730    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf"
Jul 07 14:57:54.185904 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:54.184980    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:54.351751 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:54.351660    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:54.556709 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:54.556669    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:54.617208 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:54.617132    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaafa76ab_df1f_4aba_9736_16236535b661.slice/crio-322d1756ab27b2ac99838ed512a7c22efe22d727232e3d1a51de33869585d937 WatchSource:0}: Error finding container 322d1756ab27b2ac99838ed512a7c22efe22d727232e3d1a51de33869585d937: Status 404 returned error can't find the container with id 322d1756ab27b2ac99838ed512a7c22efe22d727232e3d1a51de33869585d937
Jul 07 14:57:54.752617 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:54.752593    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"]
Jul 07 14:57:54.754177 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:54.754153    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc74be8d_e68b_416c_bf7c_231a70cd023e.slice/crio-c78c3242fedb1740b5ecc15f1d9ec690a353b42c40a76f83b1be990763a37fe2 WatchSource:0}: Error finding container c78c3242fedb1740b5ecc15f1d9ec690a353b42c40a76f83b1be990763a37fe2: Status 404 returned error can't find the container with id c78c3242fedb1740b5ecc15f1d9ec690a353b42c40a76f83b1be990763a37fe2
Jul 07 14:57:54.769644 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:54.769616    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf"]
Jul 07 14:57:54.772875 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:54.772847    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda10b510f_c07f_41d8_9f59_315b51a817fd.slice/crio-0f934524157a9171d4ddf6072f4d678ebc8310bd51b69e475854743303d13639 WatchSource:0}: Error finding container 0f934524157a9171d4ddf6072f4d678ebc8310bd51b69e475854743303d13639: Status 404 returned error can't find the container with id 0f934524157a9171d4ddf6072f4d678ebc8310bd51b69e475854743303d13639
Jul 07 14:57:54.849183 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:54.849150    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf" event={"ID":"a10b510f-c07f-41d8-9f59-315b51a817fd","Type":"ContainerStarted","Data":"0f934524157a9171d4ddf6072f4d678ebc8310bd51b69e475854743303d13639"}
Jul 07 14:57:54.850225 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:54.850193    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wxktg" event={"ID":"aafa76ab-df1f-4aba-9736-16236535b661","Type":"ContainerStarted","Data":"322d1756ab27b2ac99838ed512a7c22efe22d727232e3d1a51de33869585d937"}
Jul 07 14:57:54.851272 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:54.851239    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks" event={"ID":"bc74be8d-e68b-416c-bf7c-231a70cd023e","Type":"ContainerStarted","Data":"c78c3242fedb1740b5ecc15f1d9ec690a353b42c40a76f83b1be990763a37fe2"}
Jul 07 14:57:54.974803 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:54.974772    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:54.977107 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:54.977080    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d5b9610-2c56-40d7-a623-472a3224d65e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwztf\" (UID: \"7d5b9610-2c56-40d7-a623-472a3224d65e\") " pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:55.191089 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:55.191055    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-85x7v\""
Jul 07 14:57:55.198379 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:55.198352    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zwztf"
Jul 07 14:57:55.375558 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:55.375277    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zwztf"]
Jul 07 14:57:55.380143 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:57:55.380111    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d5b9610_2c56_40d7_a623_472a3224d65e.slice/crio-412a9cdafdb188db044d6ab0b4562d00d7da17c291d36af0cf900756098ed0e7 WatchSource:0}: Error finding container 412a9cdafdb188db044d6ab0b4562d00d7da17c291d36af0cf900756098ed0e7: Status 404 returned error can't find the container with id 412a9cdafdb188db044d6ab0b4562d00d7da17c291d36af0cf900756098ed0e7
Jul 07 14:57:55.860317 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:55.859643    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4rqtz" event={"ID":"5f99558a-08a5-4532-a131-eb30f2278bb5","Type":"ContainerStarted","Data":"ddfbcacb4f9d102417f39f705e5d66fd71f02ed1910707b9f111780e96e27c3f"}
Jul 07 14:57:55.862515 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:55.862447    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zwztf" event={"ID":"7d5b9610-2c56-40d7-a623-472a3224d65e","Type":"ContainerStarted","Data":"412a9cdafdb188db044d6ab0b4562d00d7da17c291d36af0cf900756098ed0e7"}
Jul 07 14:57:55.885445 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:55.885221    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4rqtz" podStartSLOduration=66.326934509 podStartE2EDuration="1m10.885202737s" totalImagesPullingTime="4.558268228s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:57:55.883845785 +0000 UTC m=+71.896242356" watchObservedRunningTime="2026-07-07 14:57:55.885202737 +0000 UTC m=+71.897599285"
Jul 07 14:57:57.850972 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:57.850916    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:57.955285 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:57.955236    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:58.044922 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:58.044798    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:58.634663 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:58.634615    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:59.265651 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:59.265618    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:59.367088 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:59.366988    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:59.639482 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:59.639399    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:57:59.739814 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:57:59.739777    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:01.250042 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.249983    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:01.350841 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.350750    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:01.887647 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.887555    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf" event={"ID":"a10b510f-c07f-41d8-9f59-315b51a817fd","Type":"ContainerStarted","Data":"cb49e047b7d036a730f4b19afbb342fa23ea8e36388a2535b1b45cfe9650e77c"}
Jul 07 14:58:01.889228 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.889205    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wxktg" event={"ID":"aafa76ab-df1f-4aba-9736-16236535b661","Type":"ContainerStarted","Data":"6016f38072c72f8b3ae8f4a2f8f892ba0745c51b721ade984ccb5ef77f5636cc"}
Jul 07 14:58:01.889330 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.889296    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wxktg" event={"ID":"aafa76ab-df1f-4aba-9736-16236535b661","Type":"ContainerStarted","Data":"67af290477d2c3ec0582881ed52b31e3243530bbad6f88190657f4c8da9d8c5e"}
Jul 07 14:58:01.889330 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.889320    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wxktg"
Jul 07 14:58:01.890724 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.890700    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zwztf" event={"ID":"7d5b9610-2c56-40d7-a623-472a3224d65e","Type":"ContainerStarted","Data":"283b145695174efad8720529c0d16b87f8c544885c1a94af4df70ba82db59f0e"}
Jul 07 14:58:01.892198 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.892170    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks" event={"ID":"bc74be8d-e68b-416c-bf7c-231a70cd023e","Type":"ContainerStarted","Data":"7aa8c001a8b4640bb67fbf0e1db588258642109ccd9b83ef5a02d1fc2b37f057"}
Jul 07 14:58:01.892304 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.892216    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:58:01.894000 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.893983    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks"
Jul 07 14:58:01.906592 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.906544    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6867bb98d-tz9vf" podStartSLOduration=2.6009202399999998 podStartE2EDuration="8.906529813s" totalImagesPullingTime="6.305609573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:01.905164095 +0000 UTC m=+77.917560641" watchObservedRunningTime="2026-07-07 14:58:01.906529813 +0000 UTC m=+77.918926359"
Jul 07 14:58:01.926025 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.925975    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wxktg" podStartSLOduration=36.049480904 podStartE2EDuration="41.925958302s" totalImagesPullingTime="5.876477398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:01.924714235 +0000 UTC m=+77.937110810" watchObservedRunningTime="2026-07-07 14:58:01.925958302 +0000 UTC m=+77.938354848"
Jul 07 14:58:01.942216 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:01.942167    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7646fd4cb5-gdrks" podStartSLOduration=2.285903292 podStartE2EDuration="8.942153786s" totalImagesPullingTime="6.656250494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:57:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:01.942039365 +0000 UTC m=+77.954435913" watchObservedRunningTime="2026-07-07 14:58:01.942153786 +0000 UTC m=+77.954550331"
Jul 07 14:58:02.165118 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:02.164957    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:02.649424 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:02.649394    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:02.783725 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:02.783695    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:02.897058 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:02.897017    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zwztf" event={"ID":"7d5b9610-2c56-40d7-a623-472a3224d65e","Type":"ContainerStarted","Data":"baf6eaa1b9720951bd7b31fe81e92226d02ff6bd2a1a04598a3adf3419eca5d3"}
Jul 07 14:58:03.816291 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:03.816228    2569 patch_prober.go:28] interesting pod/image-registry-6874fb6b8b-67h7c container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 07 14:58:03.816789 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:03.816322    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c" podUID="f70009d8-1c75-4b71-8836-ee0f6ec71ca9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 14:58:04.151928 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.151839    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:04.358476 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.358444    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:04.394056 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.394024    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"]
Jul 07 14:58:04.413244 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.413172    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:04.416562 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.416538    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 14:58:04.417123 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.417097    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 14:58:04.417302 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.417105    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 14:58:04.417653 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.417486    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 07 14:58:04.417653 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.417567    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 07 14:58:04.417653 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.417575    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"]
Jul 07 14:58:04.418102 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.418085    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-476f6\""
Jul 07 14:58:04.443904 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.443857    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"]
Jul 07 14:58:04.456758 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.456730    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b65zf"]
Jul 07 14:58:04.456918 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.456867    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.461034 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.461011    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 07 14:58:04.461223 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.461133    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8fr7v\""
Jul 07 14:58:04.461223 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.461209    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 07 14:58:04.462149 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.462099    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 07 14:58:04.472147 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.467546    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.472147 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.471952    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"]
Jul 07 14:58:04.476510 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.476486    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 14:58:04.476510 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.476509    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 14:58:04.476664 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.476628    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ggm5j\""
Jul 07 14:58:04.477688 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.477670    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 14:58:04.481835 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.481808    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:04.554100 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554061    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.554321 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554145    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b5883f9f-14d7-4c88-a3da-9919293e9e69-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.554321 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.554321 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554208    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0340b77d-0352-4cfd-966e-61c7c94860fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:04.554321 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554262    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.554582 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554325    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-metrics-client-ca\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.554582 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554347    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzsq5\" (UniqueName: \"kubernetes.io/projected/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-api-access-fzsq5\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.554582 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554397    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-textfile\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.554582 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554449    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s94b5\" (UniqueName: \"kubernetes.io/projected/0340b77d-0352-4cfd-966e-61c7c94860fe-kube-api-access-s94b5\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:04.554582 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554480    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0340b77d-0352-4cfd-966e-61c7c94860fe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:04.554582 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554529    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-sys\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.554582 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554556    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-root\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.554927 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554604    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5883f9f-14d7-4c88-a3da-9919293e9e69-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.554927 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554632    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-tls\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.554927 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554670    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0340b77d-0352-4cfd-966e-61c7c94860fe-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:04.554927 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554701    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-wtmp\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.554927 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554799    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.554927 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554838    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.554927 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.554864    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjjv2\" (UniqueName: \"kubernetes.io/projected/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-kube-api-access-gjjv2\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.589499 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.589459    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:04.655932 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.655897    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b5883f9f-14d7-4c88-a3da-9919293e9e69-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.656110 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.655952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656110 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656064    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0340b77d-0352-4cfd-966e-61c7c94860fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:04.656236 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656114    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.656236 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656178    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-metrics-client-ca\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656236 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:58:04.656207    2569 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 07 14:58:04.656236 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656218    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzsq5\" (UniqueName: \"kubernetes.io/projected/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-api-access-fzsq5\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.656488 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656243    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-textfile\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656488 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:58:04.656298    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0340b77d-0352-4cfd-966e-61c7c94860fe-openshift-state-metrics-tls podName:0340b77d-0352-4cfd-966e-61c7c94860fe nodeName:}" failed. No retries permitted until 2026-07-07 14:58:05.156276079 +0000 UTC m=+81.168672607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/0340b77d-0352-4cfd-966e-61c7c94860fe-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-4khdx" (UID: "0340b77d-0352-4cfd-966e-61c7c94860fe") : secret "openshift-state-metrics-tls" not found
Jul 07 14:58:04.656488 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:58:04.656309    2569 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 07 14:58:04.656488 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656337    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b5883f9f-14d7-4c88-a3da-9919293e9e69-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.656488 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:58:04.656365    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-state-metrics-tls podName:b5883f9f-14d7-4c88-a3da-9919293e9e69 nodeName:}" failed. No retries permitted until 2026-07-07 14:58:05.156347825 +0000 UTC m=+81.168744349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-fmh8f" (UID: "b5883f9f-14d7-4c88-a3da-9919293e9e69") : secret "kube-state-metrics-tls" not found
Jul 07 14:58:04.656488 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656339    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s94b5\" (UniqueName: \"kubernetes.io/projected/0340b77d-0352-4cfd-966e-61c7c94860fe-kube-api-access-s94b5\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:04.656488 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0340b77d-0352-4cfd-966e-61c7c94860fe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:04.656488 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656447    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-sys\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656488 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656472    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-root\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656499    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5883f9f-14d7-4c88-a3da-9919293e9e69-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656526    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-tls\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656553    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0340b77d-0352-4cfd-966e-61c7c94860fe-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656560    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-sys\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656576    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-wtmp\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656624    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656652    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656695    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjjv2\" (UniqueName: \"kubernetes.io/projected/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-kube-api-access-gjjv2\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656726    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:58:04.656763    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656790    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-textfile\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656826    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-wtmp\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656831    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-metrics-client-ca\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:58:04.656830    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-tls podName:b779b44e-0ea3-4708-bb55-0d8fe65e53d2 nodeName:}" failed. No retries permitted until 2026-07-07 14:58:05.156813114 +0000 UTC m=+81.169209681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-tls") pod "node-exporter-b65zf" (UID: "b779b44e-0ea3-4708-bb55-0d8fe65e53d2") : secret "node-exporter-tls" not found
Jul 07 14:58:04.656925 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.656882    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-root\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.657666 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.657180    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.657666 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.657609    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0340b77d-0352-4cfd-966e-61c7c94860fe-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:04.657666 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.657645    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5883f9f-14d7-4c88-a3da-9919293e9e69-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.658215 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.658169    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.659447 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.659425    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.659558 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.659519    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0340b77d-0352-4cfd-966e-61c7c94860fe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:04.659915 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.659893    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.668580 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.668555    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzsq5\" (UniqueName: \"kubernetes.io/projected/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-api-access-fzsq5\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:04.668701 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.668584    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjjv2\" (UniqueName: \"kubernetes.io/projected/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-kube-api-access-gjjv2\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:04.669536 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.669512    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s94b5\" (UniqueName: \"kubernetes.io/projected/0340b77d-0352-4cfd-966e-61c7c94860fe-kube-api-access-s94b5\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:04.687549 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.687509    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:04.790180 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.790144    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:04.892150 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:04.892117    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:05.069750 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.069671    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:05.161285 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.161234    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0340b77d-0352-4cfd-966e-61c7c94860fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:05.161467 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.161301    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:05.161467 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.161365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-tls\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:05.163817 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.163789    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b779b44e-0ea3-4708-bb55-0d8fe65e53d2-node-exporter-tls\") pod \"node-exporter-b65zf\" (UID: \"b779b44e-0ea3-4708-bb55-0d8fe65e53d2\") " pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:05.164216 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.164197    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0340b77d-0352-4cfd-966e-61c7c94860fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-4khdx\" (UID: \"0340b77d-0352-4cfd-966e-61c7c94860fe\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:05.164330 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.164289    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5883f9f-14d7-4c88-a3da-9919293e9e69-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fmh8f\" (UID: \"b5883f9f-14d7-4c88-a3da-9919293e9e69\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:05.263293 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.263232    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:05.326025 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.325950    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"
Jul 07 14:58:05.370958 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.370920    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"
Jul 07 14:58:05.383966 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.383938    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b65zf"
Jul 07 14:58:05.408907 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:58:05.408864    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb779b44e_0ea3_4708_bb55_0d8fe65e53d2.slice/crio-5a88603bd056fa4cab74493b6f83bcf6af8d362e090c5acebf94b2b82990996b WatchSource:0}: Error finding container 5a88603bd056fa4cab74493b6f83bcf6af8d362e090c5acebf94b2b82990996b: Status 404 returned error can't find the container with id 5a88603bd056fa4cab74493b6f83bcf6af8d362e090c5acebf94b2b82990996b
Jul 07 14:58:05.464752 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.464673    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:05.529084 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.529052    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f"]
Jul 07 14:58:05.532723 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:58:05.532692    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5883f9f_14d7_4c88_a3da_9919293e9e69.slice/crio-f70773ee3974e57df494e96cbb592c5e8949c7562920d3ee2bc2d4e6155d5ac9 WatchSource:0}: Error finding container f70773ee3974e57df494e96cbb592c5e8949c7562920d3ee2bc2d4e6155d5ac9: Status 404 returned error can't find the container with id f70773ee3974e57df494e96cbb592c5e8949c7562920d3ee2bc2d4e6155d5ac9
Jul 07 14:58:05.552016 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.551992    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx"]
Jul 07 14:58:05.555154 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:58:05.555130    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0340b77d_0352_4cfd_966e_61c7c94860fe.slice/crio-8117515d03765d74d7f275399c0abb3657a1034604719eedd3667d2ceee2b6d9 WatchSource:0}: Error finding container 8117515d03765d74d7f275399c0abb3657a1034604719eedd3667d2ceee2b6d9: Status 404 returned error can't find the container with id 8117515d03765d74d7f275399c0abb3657a1034604719eedd3667d2ceee2b6d9
Jul 07 14:58:05.574710 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.574682    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:05.790761 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.790727    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:05.861291 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.861183    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:05.910990 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.910862    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zwztf" event={"ID":"7d5b9610-2c56-40d7-a623-472a3224d65e","Type":"ContainerStarted","Data":"28cc3ce5f6befceddd34a3fea54d55ca5f1431c68f96f3ef3523cdd6489ed631"}
Jul 07 14:58:05.912986 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.912957    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx" event={"ID":"0340b77d-0352-4cfd-966e-61c7c94860fe","Type":"ContainerStarted","Data":"077294eece2d36a14b346baca5334d8f6a8b7c4124d7d324d70b64f8a2a25e89"}
Jul 07 14:58:05.913111 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.913008    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx" event={"ID":"0340b77d-0352-4cfd-966e-61c7c94860fe","Type":"ContainerStarted","Data":"5c05fa21339fdcbcd382c4a251a69b59c964026e0a6a3e7428ab108b93ebe832"}
Jul 07 14:58:05.913111 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.913030    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx" event={"ID":"0340b77d-0352-4cfd-966e-61c7c94860fe","Type":"ContainerStarted","Data":"8117515d03765d74d7f275399c0abb3657a1034604719eedd3667d2ceee2b6d9"}
Jul 07 14:58:05.914385 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.914360    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b65zf" event={"ID":"b779b44e-0ea3-4708-bb55-0d8fe65e53d2","Type":"ContainerStarted","Data":"5a88603bd056fa4cab74493b6f83bcf6af8d362e090c5acebf94b2b82990996b"}
Jul 07 14:58:05.915650 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.915629    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f" event={"ID":"b5883f9f-14d7-4c88-a3da-9919293e9e69","Type":"ContainerStarted","Data":"f70773ee3974e57df494e96cbb592c5e8949c7562920d3ee2bc2d4e6155d5ac9"}
Jul 07 14:58:05.931382 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:05.931336    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zwztf" podStartSLOduration=33.095457105 podStartE2EDuration="42.931319337s" totalImagesPullingTime="9.835862232s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:57:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:05.931293348 +0000 UTC m=+81.943689921" watchObservedRunningTime="2026-07-07 14:58:05.931319337 +0000 UTC m=+81.943715886"
Jul 07 14:58:06.921822 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:06.921786    2569 generic.go:356] "Generic (PLEG): container finished" podID="b779b44e-0ea3-4708-bb55-0d8fe65e53d2" containerID="9a031dec83054c942993dbbeae53295f8efeba6e9a56eaedd1c945a74712fa90" exitCode=0
Jul 07 14:58:06.922351 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:06.921882    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b65zf" event={"ID":"b779b44e-0ea3-4708-bb55-0d8fe65e53d2","Type":"ContainerDied","Data":"9a031dec83054c942993dbbeae53295f8efeba6e9a56eaedd1c945a74712fa90"}
Jul 07 14:58:07.378785 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.378692    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:07.480406 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.480365    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:07.578166 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.578125    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:07.648902 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.648811    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:07.732659 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.732623    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:07.927851 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.927818    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx" event={"ID":"0340b77d-0352-4cfd-966e-61c7c94860fe","Type":"ContainerStarted","Data":"44a88ac765b8f411749b5f0ab04e90269d306727975867c40e1b5ef07a3afcd3"}
Jul 07 14:58:07.929893 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.929871    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b65zf" event={"ID":"b779b44e-0ea3-4708-bb55-0d8fe65e53d2","Type":"ContainerStarted","Data":"b27b80269d45ca473a26a21c7b81ffa5589d84e6236935d8b3a51bc24875e614"}
Jul 07 14:58:07.930028 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.929910    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b65zf" event={"ID":"b779b44e-0ea3-4708-bb55-0d8fe65e53d2","Type":"ContainerStarted","Data":"5cb11c2a3af2b5bf6f3e7a231062e443c15f191fcce15f8378b75f208a3cee4e"}
Jul 07 14:58:07.931786 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.931767    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f" event={"ID":"b5883f9f-14d7-4c88-a3da-9919293e9e69","Type":"ContainerStarted","Data":"82d77e050e6594a68f35484667f84e97080a16522462854f9a8ff769cf53ddef"}
Jul 07 14:58:07.931922 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.931814    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f" event={"ID":"b5883f9f-14d7-4c88-a3da-9919293e9e69","Type":"ContainerStarted","Data":"c6215ce7ae785ebb3a78413a3edb569cec35248bd6ba2688c7ebcc42198ad748"}
Jul 07 14:58:07.931922 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.931833    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f" event={"ID":"b5883f9f-14d7-4c88-a3da-9919293e9e69","Type":"ContainerStarted","Data":"d67d642d198dc224e1541f7183b23a96dcecd2adbef3f086bcaa18484ea9d770"}
Jul 07 14:58:07.952662 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.952626    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4khdx" podStartSLOduration=2.504428604 podStartE2EDuration="3.9526133s" totalImagesPullingTime="1.448184696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:07.951097139 +0000 UTC m=+83.963493704" watchObservedRunningTime="2026-07-07 14:58:07.9526133 +0000 UTC m=+83.965009845"
Jul 07 14:58:07.984568 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:07.984525    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b65zf" podStartSLOduration=2.889977005 podStartE2EDuration="3.984512424s" totalImagesPullingTime="1.01330649s" totalInitContainerRuntime="81.228929ms" isStatefulPod=true podCreationTimestamp="2026-07-07 14:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:07.983201205 +0000 UTC m=+83.995597779" watchObservedRunningTime="2026-07-07 14:58:07.984512424 +0000 UTC m=+83.996908969"
Jul 07 14:58:08.009900 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:08.009850    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fmh8f" podStartSLOduration=2.416006526 podStartE2EDuration="4.00983764s" totalImagesPullingTime="1.593831114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:08.009123924 +0000 UTC m=+84.021520471" watchObservedRunningTime="2026-07-07 14:58:08.00983764 +0000 UTC m=+84.022234186"
Jul 07 14:58:09.303675 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:09.303636    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:09.449395 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:09.449359    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:09.519105 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:09.519071    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:09.956014 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:09.955978    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:10.056492 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.056454    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:10.637137 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.637104    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:10.688751 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.688715    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 14:58:10.692614 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.692591    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.701670 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.701623    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 07 14:58:10.701796 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.701695    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 07 14:58:10.701860 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.701819    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 07 14:58:10.701926 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.701833    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 07 14:58:10.701926 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.701908    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 07 14:58:10.702082 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.702064    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-eftaa3aim82k\""
Jul 07 14:58:10.702151 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.702073    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pj9g2\""
Jul 07 14:58:10.702213 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.702163    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 07 14:58:10.702585 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.702570    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 07 14:58:10.702679 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.702590    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 07 14:58:10.702679 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.702595    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 07 14:58:10.702679 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.702592    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 07 14:58:10.710034 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.710012    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 07 14:58:10.710157 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.710077    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 07 14:58:10.710380 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.710367    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 07 14:58:10.714382 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.714365    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 07 14:58:10.714713 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.714700    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 07 14:58:10.720691 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.720670    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 14:58:10.761348 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.761314    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:10.809511 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.809511 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809509    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.809709 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809534    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.809709 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809552    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49785725-58dd-4f67-85d6-9a3a94080dda-config-out\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.809709 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809574    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.809709 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809612    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.809709 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809671    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.809709 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809704    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.809883 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809721    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.809883 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809749    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.809883 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809778    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.809883 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809829    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-web-config\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.809883 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809868    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.810025 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809893    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.810025 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809926    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.810025 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809956    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-config\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.810025 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.809974    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49785725-58dd-4f67-85d6-9a3a94080dda-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.810025 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.810005    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.810172 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.810022    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvxwx\" (UniqueName: \"kubernetes.io/projected/49785725-58dd-4f67-85d6-9a3a94080dda-kube-api-access-tvxwx\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.810172 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.810059    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.910787 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.910699    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.910787 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.910745    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.910787 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.910771    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.910813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49785725-58dd-4f67-85d6-9a3a94080dda-config-out\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.910841    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.910865    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.910898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.910919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.910941    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.910977    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.911011    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.911038    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-web-config\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.911060    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911114 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.911092    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911672 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.911136    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911672 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.911174    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-config\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911672 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.911208    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49785725-58dd-4f67-85d6-9a3a94080dda-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911672 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.911299    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911672 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.911325    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvxwx\" (UniqueName: \"kubernetes.io/projected/49785725-58dd-4f67-85d6-9a3a94080dda-kube-api-access-tvxwx\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911672 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.911349    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.911672 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.911660    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.912024 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.911696    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.912344 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.912319    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.914603 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.913728    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49785725-58dd-4f67-85d6-9a3a94080dda-config-out\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.914603 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.913863    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.914603 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.913989    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.914603 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.914272    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.914603 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.914384    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.914603 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.914468    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.916541 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.915064    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.916541 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.915360    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.916541 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.915686    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.916541 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.915933    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.916541 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.916143    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.916541 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.916441    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49785725-58dd-4f67-85d6-9a3a94080dda-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.916541 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.916463    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-web-config\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.916914 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.916646    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49785725-58dd-4f67-85d6-9a3a94080dda-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.916914 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.916734    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.917208 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.917188    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/49785725-58dd-4f67-85d6-9a3a94080dda-config\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:10.924226 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:10.924203    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvxwx\" (UniqueName: \"kubernetes.io/projected/49785725-58dd-4f67-85d6-9a3a94080dda-kube-api-access-tvxwx\") pod \"prometheus-k8s-0\" (UID: \"49785725-58dd-4f67-85d6-9a3a94080dda\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:11.002010 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:11.001963    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:11.144807 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:11.144774    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 07 14:58:11.145749 ip-10-0-130-158 kubenswrapper[2569]: W0707 14:58:11.145724    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49785725_58dd_4f67_85d6_9a3a94080dda.slice/crio-95a4c605985c04441bc9848650054d67856738697b893b8df2c149bf5003aaad WatchSource:0}: Error finding container 95a4c605985c04441bc9848650054d67856738697b893b8df2c149bf5003aaad: Status 404 returned error can't find the container with id 95a4c605985c04441bc9848650054d67856738697b893b8df2c149bf5003aaad
Jul 07 14:58:11.899861 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:11.899833    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wxktg"
Jul 07 14:58:11.948826 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:11.948787    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49785725-58dd-4f67-85d6-9a3a94080dda","Type":"ContainerStarted","Data":"95a4c605985c04441bc9848650054d67856738697b893b8df2c149bf5003aaad"}
Jul 07 14:58:12.269289 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:12.269236    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:12.356247 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:12.356134    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:13.814916 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:13.814881    2569 patch_prober.go:28] interesting pod/image-registry-6874fb6b8b-67h7c container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 07 14:58:13.815320 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:13.814934    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c" podUID="f70009d8-1c75-4b71-8836-ee0f6ec71ca9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 14:58:13.958469 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:13.958435    2569 generic.go:356] "Generic (PLEG): container finished" podID="49785725-58dd-4f67-85d6-9a3a94080dda" containerID="410879b265c7e3da90bf0fda6b378010741e4dcb2d6b874bad207f9202b69484" exitCode=0
Jul 07 14:58:13.958612 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:13.958512    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49785725-58dd-4f67-85d6-9a3a94080dda","Type":"ContainerDied","Data":"410879b265c7e3da90bf0fda6b378010741e4dcb2d6b874bad207f9202b69484"}
Jul 07 14:58:16.387604 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:16.387556    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:16.488501 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:16.488460    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:17.391688 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:17.391660    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:17.494819 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:17.494784    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:17.977827 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:17.977789    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49785725-58dd-4f67-85d6-9a3a94080dda","Type":"ContainerStarted","Data":"f3610846599e48684ac42314d902bac4850aee5f76e00f72df98deb200ce071c"}
Jul 07 14:58:17.978011 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:17.977878    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49785725-58dd-4f67-85d6-9a3a94080dda","Type":"ContainerStarted","Data":"2010fb81a3ab14eda00607d270cd753566c11b0d725d1a71b5984140a8de456f"}
Jul 07 14:58:18.833621 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:18.833567    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c" podUID="f70009d8-1c75-4b71-8836-ee0f6ec71ca9" containerName="registry" containerID="cri-o://f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5" gracePeriod=30
Jul 07 14:58:18.964293 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:18.964230    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:19.040637 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.040510    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:19.161491 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.161466    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:58:19.184761 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.184737    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-installation-pull-secrets\") pod \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") "
Jul 07 14:58:19.184960 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.184783    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-trusted-ca\") pod \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") "
Jul 07 14:58:19.184960 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.184810    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-ca-trust-extracted\") pod \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") "
Jul 07 14:58:19.184960 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.184868    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7fkr\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-kube-api-access-x7fkr\") pod \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") "
Jul 07 14:58:19.184960 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.184893    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-bound-sa-token\") pod \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") "
Jul 07 14:58:19.184960 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.184921    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-certificates\") pod \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") "
Jul 07 14:58:19.185191 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.184983    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls\") pod \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") "
Jul 07 14:58:19.185191 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.185020    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-image-registry-private-configuration\") pod \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\" (UID: \"f70009d8-1c75-4b71-8836-ee0f6ec71ca9\") "
Jul 07 14:58:19.185304 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.185186    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f70009d8-1c75-4b71-8836-ee0f6ec71ca9" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:19.185349 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.185322    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-trusted-ca\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.186245 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.185924    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f70009d8-1c75-4b71-8836-ee0f6ec71ca9" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:58:19.189184 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.189135    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-kube-api-access-x7fkr" (OuterVolumeSpecName: "kube-api-access-x7fkr") pod "f70009d8-1c75-4b71-8836-ee0f6ec71ca9" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9"). InnerVolumeSpecName "kube-api-access-x7fkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:58:19.189682 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.189631    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f70009d8-1c75-4b71-8836-ee0f6ec71ca9" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:58:19.190343 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.190130    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f70009d8-1c75-4b71-8836-ee0f6ec71ca9" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:58:19.190343 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.190314    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f70009d8-1c75-4b71-8836-ee0f6ec71ca9" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:58:19.190732 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.190700    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f70009d8-1c75-4b71-8836-ee0f6ec71ca9" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:58:19.196763 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.196736    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f70009d8-1c75-4b71-8836-ee0f6ec71ca9" (UID: "f70009d8-1c75-4b71-8836-ee0f6ec71ca9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:58:19.286130 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.286108    2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-ca-trust-extracted\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.286130 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.286133    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7fkr\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-kube-api-access-x7fkr\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.286286 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.286142    2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-bound-sa-token\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.286286 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.286151    2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-certificates\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.286286 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.286160    2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-registry-tls\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.286286 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.286169    2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-image-registry-private-configuration\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.286286 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.286178    2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f70009d8-1c75-4b71-8836-ee0f6ec71ca9-installation-pull-secrets\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 14:58:19.807781 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.807743    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:19.945823 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.945784    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:19.986623 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.986590    2569 generic.go:356] "Generic (PLEG): container finished" podID="f70009d8-1c75-4b71-8836-ee0f6ec71ca9" containerID="f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5" exitCode=0
Jul 07 14:58:19.986787 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.986635    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c"
Jul 07 14:58:19.986787 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.986670    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c" event={"ID":"f70009d8-1c75-4b71-8836-ee0f6ec71ca9","Type":"ContainerDied","Data":"f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5"}
Jul 07 14:58:19.986787 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.986715    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6874fb6b8b-67h7c" event={"ID":"f70009d8-1c75-4b71-8836-ee0f6ec71ca9","Type":"ContainerDied","Data":"c0775fa0bdc5111d29bde862cff120fd25a4b2a060edf1ce107439275244f2f2"}
Jul 07 14:58:19.986787 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.986754    2569 scope.go:117] "RemoveContainer" containerID="f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5"
Jul 07 14:58:19.990850 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.990827    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49785725-58dd-4f67-85d6-9a3a94080dda","Type":"ContainerStarted","Data":"f7c7250e5376ff552efc4120beee4eb666903cc10a7aeb31ccc18015fb2d0e62"}
Jul 07 14:58:19.990967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.990898    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49785725-58dd-4f67-85d6-9a3a94080dda","Type":"ContainerStarted","Data":"a69f5afaa336fa78a4a7f7e358de7aa5949f1998bf10bd87ac3dd7338e39d75b"}
Jul 07 14:58:19.990967 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.990959    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49785725-58dd-4f67-85d6-9a3a94080dda","Type":"ContainerStarted","Data":"740f06244c6d46efe44f1ff0214cf3350d89431ca1b2b7b8479f1bf717ecca9a"}
Jul 07 14:58:19.991061 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.991003    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49785725-58dd-4f67-85d6-9a3a94080dda","Type":"ContainerStarted","Data":"5dc3cf66c84bfb96f08fd6d28bf638f28265252e6cdcc265bf98f77d3108b9d6"}
Jul 07 14:58:19.996656 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.996640    2569 scope.go:117] "RemoveContainer" containerID="f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5"
Jul 07 14:58:19.996908 ip-10-0-130-158 kubenswrapper[2569]: E0707 14:58:19.996889    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5\": container with ID starting with f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5 not found: ID does not exist" containerID="f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5"
Jul 07 14:58:19.996974 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:19.996915    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5"} err="failed to get container status \"f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5\": rpc error: code = NotFound desc = could not find container \"f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5\": container with ID starting with f32c607fb6a9af48f672fd578b4f6083d605b9eb422eb1aabf81a472a5870cb5 not found: ID does not exist"
Jul 07 14:58:20.021829 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:20.021792    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6874fb6b8b-67h7c"]
Jul 07 14:58:20.022958 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:20.022939    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6874fb6b8b-67h7c"]
Jul 07 14:58:20.072922 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:20.072827    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.267376605 podStartE2EDuration="10.072810722s" totalImagesPullingTime="6.787186183s" totalInitContainerRuntime="18.247934ms" isStatefulPod=false podCreationTimestamp="2026-07-07 14:58:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:20.071463476 +0000 UTC m=+96.083860052" watchObservedRunningTime="2026-07-07 14:58:20.072810722 +0000 UTC m=+96.085207268"
Jul 07 14:58:20.086502 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:20.086477    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:20.192198 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:20.192167    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:20.516354 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:20.516326    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f70009d8-1c75-4b71-8836-ee0f6ec71ca9" path="/var/lib/kubelet/pods/f70009d8-1c75-4b71-8836-ee0f6ec71ca9/volumes"
Jul 07 14:58:20.732602 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:20.732569    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:21.002311 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:21.002284    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:58:21.246366 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:21.246332    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:21.347746 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:21.347654    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:23.847623 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:23.847595    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5w2jq"
Jul 07 14:58:23.999523 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:23.999482    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:24.103980 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:24.103885    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:24.284984 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:24.284943    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:24.393798 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:24.393707    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:25.501470 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:25.501431    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:25.601669 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:25.601633    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:31.799886 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:31.799846    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:31.902823 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:31.902789    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:31.998201 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:31.998160    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:34.426744 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:34.426708    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:34.599530 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:34.599500    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:35.115997 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:35.115967    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:35.206164 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:35.206125    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:39.069169 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:39.069137    2569 generic.go:356] "Generic (PLEG): container finished" podID="d15c4fa5-ac8a-4969-9ed6-bdf1fa681576" containerID="fdf747efd00637ee94e4e4188928ca9d7e8da121d528d51d26a10f1121ac112e" exitCode=0
Jul 07 14:58:39.069584 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:39.069213    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn" event={"ID":"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576","Type":"ContainerDied","Data":"fdf747efd00637ee94e4e4188928ca9d7e8da121d528d51d26a10f1121ac112e"}
Jul 07 14:58:39.069584 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:39.069564    2569 scope.go:117] "RemoveContainer" containerID="fdf747efd00637ee94e4e4188928ca9d7e8da121d528d51d26a10f1121ac112e"
Jul 07 14:58:40.074883 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:40.074847    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-9dkqn" event={"ID":"d15c4fa5-ac8a-4969-9ed6-bdf1fa681576","Type":"ContainerStarted","Data":"0f1c91de9962d68d43e118ef3cf0d18229771529c48fdd9b0a5c56d07db203ed"}
Jul 07 14:58:52.117476 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:52.117444    2569 generic.go:356] "Generic (PLEG): container finished" podID="ae5fdbe6-b912-4300-a751-d513902c68fa" containerID="9b2d0fcc23a90848346b9a167b87bd769ffc4fc6652559007af516d5fe800752" exitCode=0
Jul 07 14:58:52.117845 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:52.117520    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm" event={"ID":"ae5fdbe6-b912-4300-a751-d513902c68fa","Type":"ContainerDied","Data":"9b2d0fcc23a90848346b9a167b87bd769ffc4fc6652559007af516d5fe800752"}
Jul 07 14:58:52.117845 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:52.117831    2569 scope.go:117] "RemoveContainer" containerID="9b2d0fcc23a90848346b9a167b87bd769ffc4fc6652559007af516d5fe800752"
Jul 07 14:58:53.124984 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:53.124949    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk7pm" event={"ID":"ae5fdbe6-b912-4300-a751-d513902c68fa","Type":"ContainerStarted","Data":"63dc031b1abd66d0f7f119d900e7b70178b837562805eeed44ce7e3d32e40327"}
Jul 07 14:58:54.184464 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:54.184428    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:58:54.257885 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:58:54.257851    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:11.002595 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:11.002561    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:59:11.022030 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:11.022000    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:59:11.212454 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:11.212430    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 14:59:23.798978 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:23.798939    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:24.028228 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:24.028184    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:32.790945 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:32.790894    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:32.870127 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:32.870092    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:32.940688 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:32.940658    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:33.527451 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:33.527422    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:39.083810 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:39.083728    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:39.158447 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:39.158408    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:50.997296 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:50.997241    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:51.057205 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:51.057173    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:53.406291 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:53.406210    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:53.513716 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:53.513685    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:53.580552 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:53.580515    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:54.188102 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:54.188065    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:54.288180 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:54.288150    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 14:59:54.382851 ip-10-0-130-158 kubenswrapper[2569]: I0707 14:59:54.382790    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:22.856274 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:00:22.856230    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:23.711924 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:00:23.711881    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:23.812165 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:00:23.812137    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:29.934157 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:00:29.934098    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:30.035562 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:00:30.035527    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:30.305591 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:00:30.305496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:30.404939 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:00:30.404900    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:30.505784 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:00:30.505750    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:51.111468 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:00:51.111432    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:51.179425 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:00:51.179393    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:58.248889 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:00:58.248854    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:00:58.356688 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:00:58.356660    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:04.094948 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:01:04.094909    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:04.482840 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:01:04.482797    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:04.595220 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:01:04.595179    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:04.738834 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:01:04.738747    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:05.027155 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:01:05.027061    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:05.705597 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:01:05.705567    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:05.798159 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:01:05.798116    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:05.916227 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:01:05.916148    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:05.997771 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:01:05.997678    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:06.453119 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:01:06.453090    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:06.554846 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:01:06.554802    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:44.394043 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:01:44.394013    2569 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 15:02:14.582784 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:02:14.582737    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:02:16.645207 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:02:16.645169    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:30.165382 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:30.165341    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:30.241954 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:30.241922    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:30.399429 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:30.399387    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:30.500225 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:30.500188    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:30.956948 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:30.956906    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:31.027931 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:31.027898    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:36.748220 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:36.748181    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:46.179308 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.179269    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-2d6jl"]
Jul 07 15:03:46.179761 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.179720    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f70009d8-1c75-4b71-8836-ee0f6ec71ca9" containerName="registry"
Jul 07 15:03:46.179761 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.179738    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f70009d8-1c75-4b71-8836-ee0f6ec71ca9" containerName="registry"
Jul 07 15:03:46.179882 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.179817    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f70009d8-1c75-4b71-8836-ee0f6ec71ca9" containerName="registry"
Jul 07 15:03:46.181759 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.181737    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-2d6jl"
Jul 07 15:03:46.184769 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.184746    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 07 15:03:46.184854 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.184775    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 07 15:03:46.185808 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.185787    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 07 15:03:46.185901 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.185789    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-lsd7p\""
Jul 07 15:03:46.192062 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.192041    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-2d6jl"]
Jul 07 15:03:46.368841 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.368803    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe39be5d-395c-4103-988f-28ce910fd01f-model-cache\") pod \"s3-init-2d6jl\" (UID: \"fe39be5d-395c-4103-988f-28ce910fd01f\") " pod="kserve/s3-init-2d6jl"
Jul 07 15:03:46.368841 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.368840    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtxnz\" (UniqueName: \"kubernetes.io/projected/fe39be5d-395c-4103-988f-28ce910fd01f-kube-api-access-mtxnz\") pod \"s3-init-2d6jl\" (UID: \"fe39be5d-395c-4103-988f-28ce910fd01f\") " pod="kserve/s3-init-2d6jl"
Jul 07 15:03:46.470190 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.470096    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe39be5d-395c-4103-988f-28ce910fd01f-model-cache\") pod \"s3-init-2d6jl\" (UID: \"fe39be5d-395c-4103-988f-28ce910fd01f\") " pod="kserve/s3-init-2d6jl"
Jul 07 15:03:46.470190 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.470144    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtxnz\" (UniqueName: \"kubernetes.io/projected/fe39be5d-395c-4103-988f-28ce910fd01f-kube-api-access-mtxnz\") pod \"s3-init-2d6jl\" (UID: \"fe39be5d-395c-4103-988f-28ce910fd01f\") " pod="kserve/s3-init-2d6jl"
Jul 07 15:03:46.470515 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.470496    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe39be5d-395c-4103-988f-28ce910fd01f-model-cache\") pod \"s3-init-2d6jl\" (UID: \"fe39be5d-395c-4103-988f-28ce910fd01f\") " pod="kserve/s3-init-2d6jl"
Jul 07 15:03:46.490743 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.490714    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtxnz\" (UniqueName: \"kubernetes.io/projected/fe39be5d-395c-4103-988f-28ce910fd01f-kube-api-access-mtxnz\") pod \"s3-init-2d6jl\" (UID: \"fe39be5d-395c-4103-988f-28ce910fd01f\") " pod="kserve/s3-init-2d6jl"
Jul 07 15:03:46.490841 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.490827    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-2d6jl"
Jul 07 15:03:46.622876 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.622843    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-2d6jl"]
Jul 07 15:03:46.626510 ip-10-0-130-158 kubenswrapper[2569]: W0707 15:03:46.626470    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe39be5d_395c_4103_988f_28ce910fd01f.slice/crio-8dcd794dd6057dd8042cb9cb6818af9ed77b0ac3bebf1ae1df3ec74b3a9ad918 WatchSource:0}: Error finding container 8dcd794dd6057dd8042cb9cb6818af9ed77b0ac3bebf1ae1df3ec74b3a9ad918: Status 404 returned error can't find the container with id 8dcd794dd6057dd8042cb9cb6818af9ed77b0ac3bebf1ae1df3ec74b3a9ad918
Jul 07 15:03:46.628409 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:46.628389    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:03:47.197883 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:47.197852    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-2d6jl" event={"ID":"fe39be5d-395c-4103-988f-28ce910fd01f","Type":"ContainerStarted","Data":"8dcd794dd6057dd8042cb9cb6818af9ed77b0ac3bebf1ae1df3ec74b3a9ad918"}
Jul 07 15:03:47.402412 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:47.402372    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:52.223489 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:52.223451    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-2d6jl" event={"ID":"fe39be5d-395c-4103-988f-28ce910fd01f","Type":"ContainerStarted","Data":"4aa5c734ac5c147f0c00bb17d2b2926a169a009bebb3130ba2337b47500b5836"}
Jul 07 15:03:53.228664 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:53.228630    2569 generic.go:356] "Generic (PLEG): container finished" podID="fe39be5d-395c-4103-988f-28ce910fd01f" containerID="4aa5c734ac5c147f0c00bb17d2b2926a169a009bebb3130ba2337b47500b5836" exitCode=0
Jul 07 15:03:53.229050 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:53.228671    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-2d6jl" event={"ID":"fe39be5d-395c-4103-988f-28ce910fd01f","Type":"ContainerDied","Data":"4aa5c734ac5c147f0c00bb17d2b2926a169a009bebb3130ba2337b47500b5836"}
Jul 07 15:03:54.519818 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:54.519754    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:54.621912 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:54.621867    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:03:59.255949 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:59.255914    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-2d6jl" event={"ID":"fe39be5d-395c-4103-988f-28ce910fd01f","Type":"ContainerStarted","Data":"6294f3e4defeae53ee403fe911d4967e93b3f0f2756f749ab8985ec1f00fa4f7"}
Jul 07 15:03:59.277075 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:03:59.277024    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-2d6jl" podStartSLOduration=1.473955796 podStartE2EDuration="13.277007847s" totalImagesPullingTime="10.098726383s" totalInitContainerRuntime="1.704325668s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:03:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:03:59.275440903 +0000 UTC m=+435.287837449" watchObservedRunningTime="2026-07-07 15:03:59.277007847 +0000 UTC m=+435.289404393"
Jul 07 15:04:01.438740 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:04:01.438698    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:04:32.436943 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:04:32.436908    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:04:32.538463 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:04:32.538431    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:04.498411 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:04.498375    2569 generic.go:356] "Generic (PLEG): container finished" podID="fe39be5d-395c-4103-988f-28ce910fd01f" containerID="6294f3e4defeae53ee403fe911d4967e93b3f0f2756f749ab8985ec1f00fa4f7" exitCode=0
Jul 07 15:05:04.498938 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:04.498450    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-2d6jl" event={"ID":"fe39be5d-395c-4103-988f-28ce910fd01f","Type":"ContainerDied","Data":"6294f3e4defeae53ee403fe911d4967e93b3f0f2756f749ab8985ec1f00fa4f7"}
Jul 07 15:05:05.629145 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:05.629124    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-2d6jl"
Jul 07 15:05:05.680929 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:05.680893    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtxnz\" (UniqueName: \"kubernetes.io/projected/fe39be5d-395c-4103-988f-28ce910fd01f-kube-api-access-mtxnz\") pod \"fe39be5d-395c-4103-988f-28ce910fd01f\" (UID: \"fe39be5d-395c-4103-988f-28ce910fd01f\") "
Jul 07 15:05:05.680929 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:05.680932    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe39be5d-395c-4103-988f-28ce910fd01f-model-cache\") pod \"fe39be5d-395c-4103-988f-28ce910fd01f\" (UID: \"fe39be5d-395c-4103-988f-28ce910fd01f\") "
Jul 07 15:05:05.683096 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:05.683064    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe39be5d-395c-4103-988f-28ce910fd01f-kube-api-access-mtxnz" (OuterVolumeSpecName: "kube-api-access-mtxnz") pod "fe39be5d-395c-4103-988f-28ce910fd01f" (UID: "fe39be5d-395c-4103-988f-28ce910fd01f"). InnerVolumeSpecName "kube-api-access-mtxnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:05:05.699991 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:05.699962    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe39be5d-395c-4103-988f-28ce910fd01f-model-cache" (OuterVolumeSpecName: "model-cache") pod "fe39be5d-395c-4103-988f-28ce910fd01f" (UID: "fe39be5d-395c-4103-988f-28ce910fd01f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:05:05.782138 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:05.782045    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtxnz\" (UniqueName: \"kubernetes.io/projected/fe39be5d-395c-4103-988f-28ce910fd01f-kube-api-access-mtxnz\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:05:05.782138 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:05.782087    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe39be5d-395c-4103-988f-28ce910fd01f-model-cache\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:05:06.506858 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:06.506821    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-2d6jl" event={"ID":"fe39be5d-395c-4103-988f-28ce910fd01f","Type":"ContainerDied","Data":"8dcd794dd6057dd8042cb9cb6818af9ed77b0ac3bebf1ae1df3ec74b3a9ad918"}
Jul 07 15:05:06.506858 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:06.506861    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8dcd794dd6057dd8042cb9cb6818af9ed77b0ac3bebf1ae1df3ec74b3a9ad918"
Jul 07 15:05:06.506858 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:06.506860    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-2d6jl"
Jul 07 15:05:06.939282 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:06.939231    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:07.020650 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:07.020618    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:19.526711 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.526669    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:19.630439 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.630400    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:19.873844 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.873752    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:19.908515 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.908487    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"]
Jul 07 15:05:19.908821 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.908807    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe39be5d-395c-4103-988f-28ce910fd01f" containerName="download-hf-model"
Jul 07 15:05:19.908864 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.908825    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe39be5d-395c-4103-988f-28ce910fd01f" containerName="download-hf-model"
Jul 07 15:05:19.908864 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.908839    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe39be5d-395c-4103-988f-28ce910fd01f" containerName="s3-init"
Jul 07 15:05:19.908864 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.908844    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe39be5d-395c-4103-988f-28ce910fd01f" containerName="s3-init"
Jul 07 15:05:19.908951 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.908893    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe39be5d-395c-4103-988f-28ce910fd01f" containerName="s3-init"
Jul 07 15:05:19.911585 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.911568    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:19.914722 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.914699    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-gzwhn\""
Jul 07 15:05:19.914957 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.914940    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 07 15:05:19.915059 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.915043    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\""
Jul 07 15:05:19.915119 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.915097    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\""
Jul 07 15:05:19.915545 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.915531    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 07 15:05:19.915625 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.915549    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 07 15:05:19.915625 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.915562    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 07 15:05:19.923690 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.923668    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"]
Jul 07 15:05:19.995224 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.995174    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8f6481c-fd34-449e-8005-872e9669d5cc-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:19.995394 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.995265    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d8f6481c-fd34-449e-8005-872e9669d5cc-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:19.995394 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.995295    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp9ll\" (UniqueName: \"kubernetes.io/projected/d8f6481c-fd34-449e-8005-872e9669d5cc-kube-api-access-tp9ll\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:19.995394 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.995314    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d8f6481c-fd34-449e-8005-872e9669d5cc-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:19.995394 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:19.995339    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d8f6481c-fd34-449e-8005-872e9669d5cc-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.008477 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.008454    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:20.096655 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.096624    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8f6481c-fd34-449e-8005-872e9669d5cc-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.096851 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.096693    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d8f6481c-fd34-449e-8005-872e9669d5cc-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.096851 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.096720    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp9ll\" (UniqueName: \"kubernetes.io/projected/d8f6481c-fd34-449e-8005-872e9669d5cc-kube-api-access-tp9ll\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.096851 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.096742    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d8f6481c-fd34-449e-8005-872e9669d5cc-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.096851 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.096775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d8f6481c-fd34-449e-8005-872e9669d5cc-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.096851 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:05:20.096784    2569 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-serving-cert: secret "isvc-sklearn-graph-1-predictor-serving-cert" not found
Jul 07 15:05:20.096851 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:05:20.096849    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8f6481c-fd34-449e-8005-872e9669d5cc-proxy-tls podName:d8f6481c-fd34-449e-8005-872e9669d5cc nodeName:}" failed. No retries permitted until 2026-07-07 15:05:20.596832126 +0000 UTC m=+516.609228664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d8f6481c-fd34-449e-8005-872e9669d5cc-proxy-tls") pod "isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" (UID: "d8f6481c-fd34-449e-8005-872e9669d5cc") : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found
Jul 07 15:05:20.097203 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.097185    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d8f6481c-fd34-449e-8005-872e9669d5cc-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.097408 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.097387    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d8f6481c-fd34-449e-8005-872e9669d5cc-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.097540 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.097522    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d8f6481c-fd34-449e-8005-872e9669d5cc-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.100300 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.100275    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:20.108470 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.108452    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp9ll\" (UniqueName: \"kubernetes.io/projected/d8f6481c-fd34-449e-8005-872e9669d5cc-kube-api-access-tp9ll\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.209847 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.209814    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:20.220042 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.220019    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"]
Jul 07 15:05:20.223821 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.223804    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.227177 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.227160    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\""
Jul 07 15:05:20.227287 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.227189    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\""
Jul 07 15:05:20.239684 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.239656    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"]
Jul 07 15:05:20.299038 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.299007    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07a6e774-7566-403d-bcba-c2399f6e7c62-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.299038 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.299040    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q44z9\" (UniqueName: \"kubernetes.io/projected/07a6e774-7566-403d-bcba-c2399f6e7c62-kube-api-access-q44z9\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.299272 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.299094    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07a6e774-7566-403d-bcba-c2399f6e7c62-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.299272 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.299142    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07a6e774-7566-403d-bcba-c2399f6e7c62-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.299272 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.299169    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07a6e774-7566-403d-bcba-c2399f6e7c62-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.312785 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.312753    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:20.400590 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.400555    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07a6e774-7566-403d-bcba-c2399f6e7c62-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.400744 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.400632    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07a6e774-7566-403d-bcba-c2399f6e7c62-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.400744 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.400665    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q44z9\" (UniqueName: \"kubernetes.io/projected/07a6e774-7566-403d-bcba-c2399f6e7c62-kube-api-access-q44z9\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.400744 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.400729    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07a6e774-7566-403d-bcba-c2399f6e7c62-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.400892 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.400771    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07a6e774-7566-403d-bcba-c2399f6e7c62-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.400926 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:05:20.400901    2569 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-xgboost-graph-predictor-serving-cert: secret "isvc-xgboost-graph-predictor-serving-cert" not found
Jul 07 15:05:20.401000 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:05:20.400982    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07a6e774-7566-403d-bcba-c2399f6e7c62-proxy-tls podName:07a6e774-7566-403d-bcba-c2399f6e7c62 nodeName:}" failed. No retries permitted until 2026-07-07 15:05:20.900949176 +0000 UTC m=+516.913345720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/07a6e774-7566-403d-bcba-c2399f6e7c62-proxy-tls") pod "isvc-xgboost-graph-predictor-555fc69d56-bd2c6" (UID: "07a6e774-7566-403d-bcba-c2399f6e7c62") : secret "isvc-xgboost-graph-predictor-serving-cert" not found
Jul 07 15:05:20.401068 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.401016    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07a6e774-7566-403d-bcba-c2399f6e7c62-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.401375 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.401353    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07a6e774-7566-403d-bcba-c2399f6e7c62-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.401448 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.401376    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07a6e774-7566-403d-bcba-c2399f6e7c62-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.417074 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.417043    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:20.419055 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.419037    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q44z9\" (UniqueName: \"kubernetes.io/projected/07a6e774-7566-403d-bcba-c2399f6e7c62-kube-api-access-q44z9\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.497751 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.497671    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:20.597913 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.597880    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:05:20.602852 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.602833    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8f6481c-fd34-449e-8005-872e9669d5cc-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.605266 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.605229    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8f6481c-fd34-449e-8005-872e9669d5cc-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.821955 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.821871    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:20.904765 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.904740    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07a6e774-7566-403d-bcba-c2399f6e7c62-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.907114 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.907090    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07a6e774-7566-403d-bcba-c2399f6e7c62-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-bd2c6\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:20.952330 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:20.952308    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"]
Jul 07 15:05:20.954804 ip-10-0-130-158 kubenswrapper[2569]: W0707 15:05:20.954774    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8f6481c_fd34_449e_8005_872e9669d5cc.slice/crio-b1f6112e2e0c425d5de7c4e458fff927e9bd2363e6e9488c3ea406516d67ec17 WatchSource:0}: Error finding container b1f6112e2e0c425d5de7c4e458fff927e9bd2363e6e9488c3ea406516d67ec17: Status 404 returned error can't find the container with id b1f6112e2e0c425d5de7c4e458fff927e9bd2363e6e9488c3ea406516d67ec17
Jul 07 15:05:21.134851 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:21.134781    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:21.272433 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:21.272401    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"]
Jul 07 15:05:21.275317 ip-10-0-130-158 kubenswrapper[2569]: W0707 15:05:21.275290    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07a6e774_7566_403d_bcba_c2399f6e7c62.slice/crio-3ca3eee57f52ae444c880ac90d86205f05750e00763df0d33bfc1e57dcb4cfe1 WatchSource:0}: Error finding container 3ca3eee57f52ae444c880ac90d86205f05750e00763df0d33bfc1e57dcb4cfe1: Status 404 returned error can't find the container with id 3ca3eee57f52ae444c880ac90d86205f05750e00763df0d33bfc1e57dcb4cfe1
Jul 07 15:05:21.563688 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:21.563649    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" event={"ID":"07a6e774-7566-403d-bcba-c2399f6e7c62","Type":"ContainerStarted","Data":"5f89469ea48fb1eda2f4e28aa22a8b0cf34b369860073518d732ef18e825d685"}
Jul 07 15:05:21.563883 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:21.563730    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" event={"ID":"07a6e774-7566-403d-bcba-c2399f6e7c62","Type":"ContainerStarted","Data":"3ca3eee57f52ae444c880ac90d86205f05750e00763df0d33bfc1e57dcb4cfe1"}
Jul 07 15:05:21.565287 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:21.565242    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" event={"ID":"d8f6481c-fd34-449e-8005-872e9669d5cc","Type":"ContainerStarted","Data":"db5a0d8151ebca3ea8803acb11ccf3fce32e7d1ce2fa32c56b1d34e020626700"}
Jul 07 15:05:21.565412 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:21.565313    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" event={"ID":"d8f6481c-fd34-449e-8005-872e9669d5cc","Type":"ContainerStarted","Data":"b1f6112e2e0c425d5de7c4e458fff927e9bd2363e6e9488c3ea406516d67ec17"}
Jul 07 15:05:24.578352 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:24.578313    2569 generic.go:356] "Generic (PLEG): container finished" podID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerID="db5a0d8151ebca3ea8803acb11ccf3fce32e7d1ce2fa32c56b1d34e020626700" exitCode=0
Jul 07 15:05:24.578721 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:24.578384    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" event={"ID":"d8f6481c-fd34-449e-8005-872e9669d5cc","Type":"ContainerDied","Data":"db5a0d8151ebca3ea8803acb11ccf3fce32e7d1ce2fa32c56b1d34e020626700"}
Jul 07 15:05:25.588053 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:25.587155    2569 generic.go:356] "Generic (PLEG): container finished" podID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerID="5f89469ea48fb1eda2f4e28aa22a8b0cf34b369860073518d732ef18e825d685" exitCode=0
Jul 07 15:05:25.588053 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:25.587304    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" event={"ID":"07a6e774-7566-403d-bcba-c2399f6e7c62","Type":"ContainerDied","Data":"5f89469ea48fb1eda2f4e28aa22a8b0cf34b369860073518d732ef18e825d685"}
Jul 07 15:05:40.678501 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:40.678466    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" event={"ID":"d8f6481c-fd34-449e-8005-872e9669d5cc","Type":"ContainerStarted","Data":"37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb"}
Jul 07 15:05:49.720909 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:49.720873    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" event={"ID":"07a6e774-7566-403d-bcba-c2399f6e7c62","Type":"ContainerStarted","Data":"dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd"}
Jul 07 15:05:51.731773 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:51.731739    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" event={"ID":"d8f6481c-fd34-449e-8005-872e9669d5cc","Type":"ContainerStarted","Data":"4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c"}
Jul 07 15:05:51.739446 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:51.739417    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:51.763900 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:51.763842    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" podStartSLOduration=2.373904227 podStartE2EDuration="32.763822351s" totalImagesPullingTime="26.890333615s" totalInitContainerRuntime="3.499584509s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:51.762045291 +0000 UTC m=+547.774441834" watchObservedRunningTime="2026-07-07 15:05:51.763822351 +0000 UTC m=+547.776218898"
Jul 07 15:05:52.738941 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:52.738908    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" event={"ID":"07a6e774-7566-403d-bcba-c2399f6e7c62","Type":"ContainerStarted","Data":"1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01"}
Jul 07 15:05:52.739436 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:52.739236    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:52.739436 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:52.739298    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:52.739436 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:52.739325    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:52.740328 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:52.740300    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 07 15:05:52.740328 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:52.740316    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 07 15:05:52.779854 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:52.779801    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" podStartSLOduration=2.958208386 podStartE2EDuration="32.779782978s" totalImagesPullingTime="25.997021721s" totalInitContainerRuntime="3.824552871s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:52.778589902 +0000 UTC m=+548.790986461" watchObservedRunningTime="2026-07-07 15:05:52.779782978 +0000 UTC m=+548.792179524"
Jul 07 15:05:53.743393 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:53.743352    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 07 15:05:53.743869 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:53.743442    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 07 15:05:58.747688 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:58.747657    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:05:58.748125 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:58.748084    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:05:58.748221 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:58.748194    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 07 15:05:58.748558 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:05:58.748532    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 07 15:06:08.748494 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:08.748457    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 07 15:06:08.748861 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:08.748461    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 07 15:06:18.748753 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:18.748714    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 07 15:06:18.749232 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:18.748723    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 07 15:06:28.749139 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:28.749098    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 07 15:06:28.749533 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:28.749098    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 07 15:06:32.472479 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:32.472443    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:32.617413 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:32.617374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:38.748882 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:38.748841    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 07 15:06:38.749288 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:38.748841    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 07 15:06:39.591415 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:39.591380    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:40.105132 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:40.105097    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:48.748749 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:48.748712    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 07 15:06:48.749130 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:48.748709    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 07 15:06:53.881686 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:53.881653    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:54.114232 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:54.114198    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:54.267015 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:54.266974    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:54.388747 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:54.388711    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:54.499866 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:54.499829    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:54.611079 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:54.610993    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:06:58.748994 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:58.748964    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:06:58.749488 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:06:58.749470    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:07:19.727616 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:19.725357    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"]
Jul 07 15:07:19.729455 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:19.729015    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:07:19.732112 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:19.732089    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\""
Jul 07 15:07:19.732223 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:19.732185    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\""
Jul 07 15:07:19.737160 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:19.737137    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:19.744209 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:19.744185    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"]
Jul 07 15:07:19.829199 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:19.829169    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5623fdcd-4046-4c6b-8998-ad5e92d1880d-openshift-service-ca-bundle\") pod \"model-chainer-b8bd544f6-6mjp5\" (UID: \"5623fdcd-4046-4c6b-8998-ad5e92d1880d\") " pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:07:19.829199 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:19.829204    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5623fdcd-4046-4c6b-8998-ad5e92d1880d-proxy-tls\") pod \"model-chainer-b8bd544f6-6mjp5\" (UID: \"5623fdcd-4046-4c6b-8998-ad5e92d1880d\") " pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:07:19.930140 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:19.930087    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5623fdcd-4046-4c6b-8998-ad5e92d1880d-openshift-service-ca-bundle\") pod \"model-chainer-b8bd544f6-6mjp5\" (UID: \"5623fdcd-4046-4c6b-8998-ad5e92d1880d\") " pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:07:19.930140 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:19.930149    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5623fdcd-4046-4c6b-8998-ad5e92d1880d-proxy-tls\") pod \"model-chainer-b8bd544f6-6mjp5\" (UID: \"5623fdcd-4046-4c6b-8998-ad5e92d1880d\") " pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:07:19.930386 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:07:19.930327    2569 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found
Jul 07 15:07:19.930423 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:07:19.930413    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5623fdcd-4046-4c6b-8998-ad5e92d1880d-proxy-tls podName:5623fdcd-4046-4c6b-8998-ad5e92d1880d nodeName:}" failed. No retries permitted until 2026-07-07 15:07:20.430395667 +0000 UTC m=+636.442792191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/5623fdcd-4046-4c6b-8998-ad5e92d1880d-proxy-tls") pod "model-chainer-b8bd544f6-6mjp5" (UID: "5623fdcd-4046-4c6b-8998-ad5e92d1880d") : secret "model-chainer-serving-cert" not found
Jul 07 15:07:19.930743 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:19.930722    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5623fdcd-4046-4c6b-8998-ad5e92d1880d-openshift-service-ca-bundle\") pod \"model-chainer-b8bd544f6-6mjp5\" (UID: \"5623fdcd-4046-4c6b-8998-ad5e92d1880d\") " pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:07:20.434436 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:20.434398    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5623fdcd-4046-4c6b-8998-ad5e92d1880d-proxy-tls\") pod \"model-chainer-b8bd544f6-6mjp5\" (UID: \"5623fdcd-4046-4c6b-8998-ad5e92d1880d\") " pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:07:20.436777 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:20.436753    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5623fdcd-4046-4c6b-8998-ad5e92d1880d-proxy-tls\") pod \"model-chainer-b8bd544f6-6mjp5\" (UID: \"5623fdcd-4046-4c6b-8998-ad5e92d1880d\") " pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:07:20.639794 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:20.639761    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:07:20.773009 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:20.772982    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"]
Jul 07 15:07:20.775118 ip-10-0-130-158 kubenswrapper[2569]: W0707 15:07:20.775087    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5623fdcd_4046_4c6b_8998_ad5e92d1880d.slice/crio-52b005f544c2cd152b162a87d3d495ffb8db45399ff25e906749206e35e5c94f WatchSource:0}: Error finding container 52b005f544c2cd152b162a87d3d495ffb8db45399ff25e906749206e35e5c94f: Status 404 returned error can't find the container with id 52b005f544c2cd152b162a87d3d495ffb8db45399ff25e906749206e35e5c94f
Jul 07 15:07:21.086391 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:21.086308    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5" event={"ID":"5623fdcd-4046-4c6b-8998-ad5e92d1880d","Type":"ContainerStarted","Data":"52b005f544c2cd152b162a87d3d495ffb8db45399ff25e906749206e35e5c94f"}
Jul 07 15:07:22.053240 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:22.053195    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:23.095729 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:23.095695    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5" event={"ID":"5623fdcd-4046-4c6b-8998-ad5e92d1880d","Type":"ContainerStarted","Data":"f06217649618786df2139cb95421d1ef7835b068c6ec07b60436affb2e75a173"}
Jul 07 15:07:23.096122 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:23.095826    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:07:23.121947 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:23.121889    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5" podStartSLOduration=1.964534432 podStartE2EDuration="4.121874552s" totalImagesPullingTime="2.15734012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:07:23.121143093 +0000 UTC m=+639.133539638" watchObservedRunningTime="2026-07-07 15:07:23.121874552 +0000 UTC m=+639.134271098"
Jul 07 15:07:29.106347 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:29.106317    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:07:29.770175 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:29.770141    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"]
Jul 07 15:07:29.770400 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:29.770372    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5" podUID="5623fdcd-4046-4c6b-8998-ad5e92d1880d" containerName="model-chainer" containerID="cri-o://f06217649618786df2139cb95421d1ef7835b068c6ec07b60436affb2e75a173" gracePeriod=30
Jul 07 15:07:29.810720 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:29.810682    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:29.981073 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:29.981042    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"]
Jul 07 15:07:29.981343 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:29.981302    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kserve-container" containerID="cri-o://37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb" gracePeriod=30
Jul 07 15:07:29.981432 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:29.981406    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kube-rbac-proxy" containerID="cri-o://4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c" gracePeriod=30
Jul 07 15:07:30.062694 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:30.062601    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:30.126246 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:30.126215    2569 generic.go:356] "Generic (PLEG): container finished" podID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerID="4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c" exitCode=2
Jul 07 15:07:30.126703 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:30.126289    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" event={"ID":"d8f6481c-fd34-449e-8005-872e9669d5cc","Type":"ContainerDied","Data":"4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c"}
Jul 07 15:07:30.177135 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:30.177095    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:30.252358 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:30.252331    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"]
Jul 07 15:07:30.252566 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:30.252545    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kserve-container" containerID="cri-o://dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd" gracePeriod=30
Jul 07 15:07:30.252645 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:30.252578    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kube-rbac-proxy" containerID="cri-o://1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01" gracePeriod=30
Jul 07 15:07:30.256486 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:30.256453    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:30.353796 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:30.353713    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:30.522636 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:30.522591    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:31.132919 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:31.132882    2569 generic.go:356] "Generic (PLEG): container finished" podID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerID="1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01" exitCode=2
Jul 07 15:07:31.133306 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:31.132965    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" event={"ID":"07a6e774-7566-403d-bcba-c2399f6e7c62","Type":"ContainerDied","Data":"1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01"}
Jul 07 15:07:33.744511 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.744475    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.20:8643/healthz\": dial tcp 10.133.0.20:8643: connect: connection refused"
Jul 07 15:07:33.744511 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.744489    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused"
Jul 07 15:07:33.893646 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.893622    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:07:33.949359 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.949328    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q44z9\" (UniqueName: \"kubernetes.io/projected/07a6e774-7566-403d-bcba-c2399f6e7c62-kube-api-access-q44z9\") pod \"07a6e774-7566-403d-bcba-c2399f6e7c62\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") "
Jul 07 15:07:33.949359 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.949373    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07a6e774-7566-403d-bcba-c2399f6e7c62-kserve-provision-location\") pod \"07a6e774-7566-403d-bcba-c2399f6e7c62\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") "
Jul 07 15:07:33.949623 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.949414    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07a6e774-7566-403d-bcba-c2399f6e7c62-cabundle-cert\") pod \"07a6e774-7566-403d-bcba-c2399f6e7c62\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") "
Jul 07 15:07:33.949623 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.949453    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07a6e774-7566-403d-bcba-c2399f6e7c62-proxy-tls\") pod \"07a6e774-7566-403d-bcba-c2399f6e7c62\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") "
Jul 07 15:07:33.949623 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.949509    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07a6e774-7566-403d-bcba-c2399f6e7c62-kube-rbac-proxy-sar-config\") pod \"07a6e774-7566-403d-bcba-c2399f6e7c62\" (UID: \"07a6e774-7566-403d-bcba-c2399f6e7c62\") "
Jul 07 15:07:33.949856 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.949828    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07a6e774-7566-403d-bcba-c2399f6e7c62-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "07a6e774-7566-403d-bcba-c2399f6e7c62" (UID: "07a6e774-7566-403d-bcba-c2399f6e7c62"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:07:33.949856 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.949850    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07a6e774-7566-403d-bcba-c2399f6e7c62-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "07a6e774-7566-403d-bcba-c2399f6e7c62" (UID: "07a6e774-7566-403d-bcba-c2399f6e7c62"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:07:33.949951 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.949895    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07a6e774-7566-403d-bcba-c2399f6e7c62-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "07a6e774-7566-403d-bcba-c2399f6e7c62" (UID: "07a6e774-7566-403d-bcba-c2399f6e7c62"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:07:33.951626 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.951602    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07a6e774-7566-403d-bcba-c2399f6e7c62-kube-api-access-q44z9" (OuterVolumeSpecName: "kube-api-access-q44z9") pod "07a6e774-7566-403d-bcba-c2399f6e7c62" (UID: "07a6e774-7566-403d-bcba-c2399f6e7c62"). InnerVolumeSpecName "kube-api-access-q44z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:07:33.951735 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:33.951662    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07a6e774-7566-403d-bcba-c2399f6e7c62-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "07a6e774-7566-403d-bcba-c2399f6e7c62" (UID: "07a6e774-7566-403d-bcba-c2399f6e7c62"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:07:34.050289 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.050176    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07a6e774-7566-403d-bcba-c2399f6e7c62-kserve-provision-location\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:07:34.050289 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.050208    2569 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07a6e774-7566-403d-bcba-c2399f6e7c62-cabundle-cert\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:07:34.050289 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.050219    2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07a6e774-7566-403d-bcba-c2399f6e7c62-proxy-tls\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:07:34.050289 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.050227    2569 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07a6e774-7566-403d-bcba-c2399f6e7c62-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:07:34.050289 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.050237    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q44z9\" (UniqueName: \"kubernetes.io/projected/07a6e774-7566-403d-bcba-c2399f6e7c62-kube-api-access-q44z9\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:07:34.104316 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.104243    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5" podUID="5623fdcd-4046-4c6b-8998-ad5e92d1880d" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:34.145858 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.145825    2569 generic.go:356] "Generic (PLEG): container finished" podID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerID="dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd" exitCode=0
Jul 07 15:07:34.146014 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.145890    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"
Jul 07 15:07:34.146014 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.145896    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" event={"ID":"07a6e774-7566-403d-bcba-c2399f6e7c62","Type":"ContainerDied","Data":"dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd"}
Jul 07 15:07:34.146014 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.145959    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6" event={"ID":"07a6e774-7566-403d-bcba-c2399f6e7c62","Type":"ContainerDied","Data":"3ca3eee57f52ae444c880ac90d86205f05750e00763df0d33bfc1e57dcb4cfe1"}
Jul 07 15:07:34.146014 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.145991    2569 scope.go:117] "RemoveContainer" containerID="1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01"
Jul 07 15:07:34.154688 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.154667    2569 scope.go:117] "RemoveContainer" containerID="dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd"
Jul 07 15:07:34.161974 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.161959    2569 scope.go:117] "RemoveContainer" containerID="5f89469ea48fb1eda2f4e28aa22a8b0cf34b369860073518d732ef18e825d685"
Jul 07 15:07:34.169225 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.169197    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"]
Jul 07 15:07:34.170461 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.170443    2569 scope.go:117] "RemoveContainer" containerID="1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01"
Jul 07 15:07:34.170776 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:07:34.170749    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01\": container with ID starting with 1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01 not found: ID does not exist" containerID="1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01"
Jul 07 15:07:34.170864 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.170783    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01"} err="failed to get container status \"1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01\": rpc error: code = NotFound desc = could not find container \"1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01\": container with ID starting with 1e4440ac00f4bcd10d9f780cf0fc167a8b0221ea6525cf7610fd75350c1fae01 not found: ID does not exist"
Jul 07 15:07:34.170864 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.170806    2569 scope.go:117] "RemoveContainer" containerID="dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd"
Jul 07 15:07:34.171086 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:07:34.171058    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd\": container with ID starting with dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd not found: ID does not exist" containerID="dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd"
Jul 07 15:07:34.171152 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.171095    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd"} err="failed to get container status \"dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd\": rpc error: code = NotFound desc = could not find container \"dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd\": container with ID starting with dca065264fe46ef7b9b577fd9b5550777cdedbc29d7456ffca073c112cb00bfd not found: ID does not exist"
Jul 07 15:07:34.171152 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.171117    2569 scope.go:117] "RemoveContainer" containerID="5f89469ea48fb1eda2f4e28aa22a8b0cf34b369860073518d732ef18e825d685"
Jul 07 15:07:34.171447 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:07:34.171427    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f89469ea48fb1eda2f4e28aa22a8b0cf34b369860073518d732ef18e825d685\": container with ID starting with 5f89469ea48fb1eda2f4e28aa22a8b0cf34b369860073518d732ef18e825d685 not found: ID does not exist" containerID="5f89469ea48fb1eda2f4e28aa22a8b0cf34b369860073518d732ef18e825d685"
Jul 07 15:07:34.171525 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.171453    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f89469ea48fb1eda2f4e28aa22a8b0cf34b369860073518d732ef18e825d685"} err="failed to get container status \"5f89469ea48fb1eda2f4e28aa22a8b0cf34b369860073518d732ef18e825d685\": rpc error: code = NotFound desc = could not find container \"5f89469ea48fb1eda2f4e28aa22a8b0cf34b369860073518d732ef18e825d685\": container with ID starting with 5f89469ea48fb1eda2f4e28aa22a8b0cf34b369860073518d732ef18e825d685 not found: ID does not exist"
Jul 07 15:07:34.172053 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.172036    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-bd2c6"]
Jul 07 15:07:34.516054 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.516024    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" path="/var/lib/kubelet/pods/07a6e774-7566-403d-bcba-c2399f6e7c62/volumes"
Jul 07 15:07:34.827012 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.826989    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:07:34.855738 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.855709    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d8f6481c-fd34-449e-8005-872e9669d5cc-kube-rbac-proxy-sar-config\") pod \"d8f6481c-fd34-449e-8005-872e9669d5cc\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") "
Jul 07 15:07:34.855900 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.855759    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp9ll\" (UniqueName: \"kubernetes.io/projected/d8f6481c-fd34-449e-8005-872e9669d5cc-kube-api-access-tp9ll\") pod \"d8f6481c-fd34-449e-8005-872e9669d5cc\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") "
Jul 07 15:07:34.855900 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.855833    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d8f6481c-fd34-449e-8005-872e9669d5cc-cabundle-cert\") pod \"d8f6481c-fd34-449e-8005-872e9669d5cc\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") "
Jul 07 15:07:34.855900 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.855868    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8f6481c-fd34-449e-8005-872e9669d5cc-proxy-tls\") pod \"d8f6481c-fd34-449e-8005-872e9669d5cc\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") "
Jul 07 15:07:34.856056 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.855904    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d8f6481c-fd34-449e-8005-872e9669d5cc-kserve-provision-location\") pod \"d8f6481c-fd34-449e-8005-872e9669d5cc\" (UID: \"d8f6481c-fd34-449e-8005-872e9669d5cc\") "
Jul 07 15:07:34.856336 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.856306    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8f6481c-fd34-449e-8005-872e9669d5cc-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d8f6481c-fd34-449e-8005-872e9669d5cc" (UID: "d8f6481c-fd34-449e-8005-872e9669d5cc"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:07:34.856470 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.856361    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8f6481c-fd34-449e-8005-872e9669d5cc-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d8f6481c-fd34-449e-8005-872e9669d5cc" (UID: "d8f6481c-fd34-449e-8005-872e9669d5cc"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:07:34.856470 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.856431    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8f6481c-fd34-449e-8005-872e9669d5cc-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "d8f6481c-fd34-449e-8005-872e9669d5cc" (UID: "d8f6481c-fd34-449e-8005-872e9669d5cc"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:07:34.858031 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.858001    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8f6481c-fd34-449e-8005-872e9669d5cc-kube-api-access-tp9ll" (OuterVolumeSpecName: "kube-api-access-tp9ll") pod "d8f6481c-fd34-449e-8005-872e9669d5cc" (UID: "d8f6481c-fd34-449e-8005-872e9669d5cc"). InnerVolumeSpecName "kube-api-access-tp9ll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:07:34.858307 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.858282    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8f6481c-fd34-449e-8005-872e9669d5cc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d8f6481c-fd34-449e-8005-872e9669d5cc" (UID: "d8f6481c-fd34-449e-8005-872e9669d5cc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:07:34.956734 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.956696    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp9ll\" (UniqueName: \"kubernetes.io/projected/d8f6481c-fd34-449e-8005-872e9669d5cc-kube-api-access-tp9ll\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:07:34.956734 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.956729    2569 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d8f6481c-fd34-449e-8005-872e9669d5cc-cabundle-cert\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:07:34.956734 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.956739    2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8f6481c-fd34-449e-8005-872e9669d5cc-proxy-tls\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:07:34.956969 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.956748    2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d8f6481c-fd34-449e-8005-872e9669d5cc-kserve-provision-location\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:07:34.956969 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:34.956757    2569 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d8f6481c-fd34-449e-8005-872e9669d5cc-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:07:35.152475 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.152394    2569 generic.go:356] "Generic (PLEG): container finished" podID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerID="37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb" exitCode=0
Jul 07 15:07:35.152475 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.152457    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"
Jul 07 15:07:35.152661 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.152471    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" event={"ID":"d8f6481c-fd34-449e-8005-872e9669d5cc","Type":"ContainerDied","Data":"37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb"}
Jul 07 15:07:35.152661 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.152529    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk" event={"ID":"d8f6481c-fd34-449e-8005-872e9669d5cc","Type":"ContainerDied","Data":"b1f6112e2e0c425d5de7c4e458fff927e9bd2363e6e9488c3ea406516d67ec17"}
Jul 07 15:07:35.152661 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.152555    2569 scope.go:117] "RemoveContainer" containerID="4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c"
Jul 07 15:07:35.165840 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.165822    2569 scope.go:117] "RemoveContainer" containerID="37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb"
Jul 07 15:07:35.173056 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.173037    2569 scope.go:117] "RemoveContainer" containerID="db5a0d8151ebca3ea8803acb11ccf3fce32e7d1ce2fa32c56b1d34e020626700"
Jul 07 15:07:35.180299 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.180274    2569 scope.go:117] "RemoveContainer" containerID="4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c"
Jul 07 15:07:35.180586 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:07:35.180563    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c\": container with ID starting with 4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c not found: ID does not exist" containerID="4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c"
Jul 07 15:07:35.180637 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.180597    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c"} err="failed to get container status \"4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c\": rpc error: code = NotFound desc = could not find container \"4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c\": container with ID starting with 4ff9f3d5bef8d1d7761f0b2aea9a54c3439690642fb3c70ed18b1bc342ec8b6c not found: ID does not exist"
Jul 07 15:07:35.180637 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.180622    2569 scope.go:117] "RemoveContainer" containerID="37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb"
Jul 07 15:07:35.180909 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:07:35.180891    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb\": container with ID starting with 37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb not found: ID does not exist" containerID="37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb"
Jul 07 15:07:35.180946 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.180915    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb"} err="failed to get container status \"37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb\": rpc error: code = NotFound desc = could not find container \"37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb\": container with ID starting with 37bb37dffbc45d41aaa1efb15f14b23ee3c9f875ce9e5a9a1ec7a955456552fb not found: ID does not exist"
Jul 07 15:07:35.180946 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.180936    2569 scope.go:117] "RemoveContainer" containerID="db5a0d8151ebca3ea8803acb11ccf3fce32e7d1ce2fa32c56b1d34e020626700"
Jul 07 15:07:35.181160 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.181139    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"]
Jul 07 15:07:35.181246 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:07:35.181169    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db5a0d8151ebca3ea8803acb11ccf3fce32e7d1ce2fa32c56b1d34e020626700\": container with ID starting with db5a0d8151ebca3ea8803acb11ccf3fce32e7d1ce2fa32c56b1d34e020626700 not found: ID does not exist" containerID="db5a0d8151ebca3ea8803acb11ccf3fce32e7d1ce2fa32c56b1d34e020626700"
Jul 07 15:07:35.181246 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.181187    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db5a0d8151ebca3ea8803acb11ccf3fce32e7d1ce2fa32c56b1d34e020626700"} err="failed to get container status \"db5a0d8151ebca3ea8803acb11ccf3fce32e7d1ce2fa32c56b1d34e020626700\": rpc error: code = NotFound desc = could not find container \"db5a0d8151ebca3ea8803acb11ccf3fce32e7d1ce2fa32c56b1d34e020626700\": container with ID starting with db5a0d8151ebca3ea8803acb11ccf3fce32e7d1ce2fa32c56b1d34e020626700 not found: ID does not exist"
Jul 07 15:07:35.186217 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:35.186196    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-766fdfb6bb-wbdsk"]
Jul 07 15:07:36.516331 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:36.516298    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" path="/var/lib/kubelet/pods/d8f6481c-fd34-449e-8005-872e9669d5cc/volumes"
Jul 07 15:07:39.104714 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:39.104671    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5" podUID="5623fdcd-4046-4c6b-8998-ad5e92d1880d" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:44.104523 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:44.104481    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5" podUID="5623fdcd-4046-4c6b-8998-ad5e92d1880d" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:44.104924 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:44.104611    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:07:49.104134 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:49.104097    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5" podUID="5623fdcd-4046-4c6b-8998-ad5e92d1880d" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:54.104923 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:54.104885    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5" podUID="5623fdcd-4046-4c6b-8998-ad5e92d1880d" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:07:54.118410 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:54.118377    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:54.273955 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:54.273923    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:07:59.104349 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:07:59.104305    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5" podUID="5623fdcd-4046-4c6b-8998-ad5e92d1880d" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:08:00.252417 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:00.252379    2569 generic.go:356] "Generic (PLEG): container finished" podID="5623fdcd-4046-4c6b-8998-ad5e92d1880d" containerID="f06217649618786df2139cb95421d1ef7835b068c6ec07b60436affb2e75a173" exitCode=0
Jul 07 15:08:00.252856 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:00.252431    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5" event={"ID":"5623fdcd-4046-4c6b-8998-ad5e92d1880d","Type":"ContainerDied","Data":"f06217649618786df2139cb95421d1ef7835b068c6ec07b60436affb2e75a173"}
Jul 07 15:08:00.417845 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:00.417818    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:08:00.466623 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:00.466594    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5623fdcd-4046-4c6b-8998-ad5e92d1880d-openshift-service-ca-bundle\") pod \"5623fdcd-4046-4c6b-8998-ad5e92d1880d\" (UID: \"5623fdcd-4046-4c6b-8998-ad5e92d1880d\") "
Jul 07 15:08:00.466772 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:00.466668    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5623fdcd-4046-4c6b-8998-ad5e92d1880d-proxy-tls\") pod \"5623fdcd-4046-4c6b-8998-ad5e92d1880d\" (UID: \"5623fdcd-4046-4c6b-8998-ad5e92d1880d\") "
Jul 07 15:08:00.466967 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:00.466946    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5623fdcd-4046-4c6b-8998-ad5e92d1880d-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "5623fdcd-4046-4c6b-8998-ad5e92d1880d" (UID: "5623fdcd-4046-4c6b-8998-ad5e92d1880d"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:08:00.468626 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:00.468602    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5623fdcd-4046-4c6b-8998-ad5e92d1880d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5623fdcd-4046-4c6b-8998-ad5e92d1880d" (UID: "5623fdcd-4046-4c6b-8998-ad5e92d1880d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:08:00.567463 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:00.567436    2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5623fdcd-4046-4c6b-8998-ad5e92d1880d-proxy-tls\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:08:00.567463 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:00.567460    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5623fdcd-4046-4c6b-8998-ad5e92d1880d-openshift-service-ca-bundle\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:08:01.257541 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:01.257513    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"
Jul 07 15:08:01.257541 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:01.257527    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5" event={"ID":"5623fdcd-4046-4c6b-8998-ad5e92d1880d","Type":"ContainerDied","Data":"52b005f544c2cd152b162a87d3d495ffb8db45399ff25e906749206e35e5c94f"}
Jul 07 15:08:01.258036 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:01.257589    2569 scope.go:117] "RemoveContainer" containerID="f06217649618786df2139cb95421d1ef7835b068c6ec07b60436affb2e75a173"
Jul 07 15:08:01.302315 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:01.302279    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"]
Jul 07 15:08:01.307435 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:01.307410    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-b8bd544f6-6mjp5"]
Jul 07 15:08:02.516817 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:02.516782    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5623fdcd-4046-4c6b-8998-ad5e92d1880d" path="/var/lib/kubelet/pods/5623fdcd-4046-4c6b-8998-ad5e92d1880d/volumes"
Jul 07 15:08:29.983183 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.983136    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:08:29.984790 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.984766    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"]
Jul 07 15:08:29.985046 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985033    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="storage-initializer"
Jul 07 15:08:29.985103 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985047    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="storage-initializer"
Jul 07 15:08:29.985103 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985059    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kube-rbac-proxy"
Jul 07 15:08:29.985103 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985065    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kube-rbac-proxy"
Jul 07 15:08:29.985103 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985073    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kserve-container"
Jul 07 15:08:29.985103 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985079    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kserve-container"
Jul 07 15:08:29.985103 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985087    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="storage-initializer"
Jul 07 15:08:29.985103 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985093    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="storage-initializer"
Jul 07 15:08:29.985103 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985102    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5623fdcd-4046-4c6b-8998-ad5e92d1880d" containerName="model-chainer"
Jul 07 15:08:29.985103 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985107    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5623fdcd-4046-4c6b-8998-ad5e92d1880d" containerName="model-chainer"
Jul 07 15:08:29.985591 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985114    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kserve-container"
Jul 07 15:08:29.985591 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985120    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kserve-container"
Jul 07 15:08:29.985591 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985129    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kube-rbac-proxy"
Jul 07 15:08:29.985591 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985134    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kube-rbac-proxy"
Jul 07 15:08:29.985591 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985177    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kube-rbac-proxy"
Jul 07 15:08:29.985591 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985185    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5623fdcd-4046-4c6b-8998-ad5e92d1880d" containerName="model-chainer"
Jul 07 15:08:29.985591 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985192    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8f6481c-fd34-449e-8005-872e9669d5cc" containerName="kserve-container"
Jul 07 15:08:29.985591 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985198    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kserve-container"
Jul 07 15:08:29.985591 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.985203    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="07a6e774-7566-403d-bcba-c2399f6e7c62" containerName="kube-rbac-proxy"
Jul 07 15:08:29.987921 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.987901    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:08:29.990736 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.990719    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-8996c-kube-rbac-proxy-sar-config\""
Jul 07 15:08:29.990886 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.990869    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-8996c-serving-cert\""
Jul 07 15:08:29.990964 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.990950    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 07 15:08:29.991024 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:29.990999    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-gzwhn\""
Jul 07 15:08:30.000123 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:30.000105    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"]
Jul 07 15:08:30.009701 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:30.009674    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b38ebfe9-843d-4965-8da5-7fc858a0b4eb-openshift-service-ca-bundle\") pod \"sequence-graph-8996c-6b8f8959c8-dl6dm\" (UID: \"b38ebfe9-843d-4965-8da5-7fc858a0b4eb\") " pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:08:30.009812 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:30.009730    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b38ebfe9-843d-4965-8da5-7fc858a0b4eb-proxy-tls\") pod \"sequence-graph-8996c-6b8f8959c8-dl6dm\" (UID: \"b38ebfe9-843d-4965-8da5-7fc858a0b4eb\") " pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:08:30.110579 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:30.110543    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b38ebfe9-843d-4965-8da5-7fc858a0b4eb-openshift-service-ca-bundle\") pod \"sequence-graph-8996c-6b8f8959c8-dl6dm\" (UID: \"b38ebfe9-843d-4965-8da5-7fc858a0b4eb\") " pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:08:30.110579 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:30.110588    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b38ebfe9-843d-4965-8da5-7fc858a0b4eb-proxy-tls\") pod \"sequence-graph-8996c-6b8f8959c8-dl6dm\" (UID: \"b38ebfe9-843d-4965-8da5-7fc858a0b4eb\") " pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:08:30.111174 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:30.111155    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b38ebfe9-843d-4965-8da5-7fc858a0b4eb-openshift-service-ca-bundle\") pod \"sequence-graph-8996c-6b8f8959c8-dl6dm\" (UID: \"b38ebfe9-843d-4965-8da5-7fc858a0b4eb\") " pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:08:30.113046 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:30.113022    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b38ebfe9-843d-4965-8da5-7fc858a0b4eb-proxy-tls\") pod \"sequence-graph-8996c-6b8f8959c8-dl6dm\" (UID: \"b38ebfe9-843d-4965-8da5-7fc858a0b4eb\") " pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:08:30.156984 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:30.156948    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:08:30.298360 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:30.298269    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:08:30.431325 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:30.431202    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"]
Jul 07 15:08:30.433807 ip-10-0-130-158 kubenswrapper[2569]: W0707 15:08:30.433784    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb38ebfe9_843d_4965_8da5_7fc858a0b4eb.slice/crio-202ed4b63e831a592d18b383a698e1f9fa220bfcbb3ff8842a14e5efcdd231c8 WatchSource:0}: Error finding container 202ed4b63e831a592d18b383a698e1f9fa220bfcbb3ff8842a14e5efcdd231c8: Status 404 returned error can't find the container with id 202ed4b63e831a592d18b383a698e1f9fa220bfcbb3ff8842a14e5efcdd231c8
Jul 07 15:08:31.370989 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:31.370958    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm" event={"ID":"b38ebfe9-843d-4965-8da5-7fc858a0b4eb","Type":"ContainerStarted","Data":"49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957"}
Jul 07 15:08:31.371408 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:31.371008    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm" event={"ID":"b38ebfe9-843d-4965-8da5-7fc858a0b4eb","Type":"ContainerStarted","Data":"202ed4b63e831a592d18b383a698e1f9fa220bfcbb3ff8842a14e5efcdd231c8"}
Jul 07 15:08:31.371408 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:31.371049    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:08:31.396344 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:31.396290    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm" podStartSLOduration=2.396272807 podStartE2EDuration="2.396272807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:08:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:08:31.395802171 +0000 UTC m=+707.408198719" watchObservedRunningTime="2026-07-07 15:08:31.396272807 +0000 UTC m=+707.408669345"
Jul 07 15:08:34.471555 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:34.471520    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:08:37.379972 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:37.379897    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:08:43.564951 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:08:43.564900    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:09:35.983055 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:09:35.983009    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:11:43.369603 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:11:43.369517    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:16:08.986160 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:08.986119    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:16:10.455095 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:10.455057    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:16:44.697307 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:44.697275    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"]
Jul 07 15:16:44.697785 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:44.697465    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm" podUID="b38ebfe9-843d-4965-8da5-7fc858a0b4eb" containerName="sequence-graph-8996c" containerID="cri-o://49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957" gracePeriod=30
Jul 07 15:16:44.727948 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:44.727852    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:16:44.844394 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:44.844360    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:16:44.972324 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:44.972169    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:16:45.104429 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:45.104389    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:16:45.199205 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:45.199119    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:16:47.378889 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:47.378852    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm" podUID="b38ebfe9-843d-4965-8da5-7fc858a0b4eb" containerName="sequence-graph-8996c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:16:50.844548 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:50.844518    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:16:52.378813 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:52.378776    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm" podUID="b38ebfe9-843d-4965-8da5-7fc858a0b4eb" containerName="sequence-graph-8996c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:16:57.378929 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:57.378892    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm" podUID="b38ebfe9-843d-4965-8da5-7fc858a0b4eb" containerName="sequence-graph-8996c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:16:57.379330 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:16:57.379010    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:17:02.378547 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:02.378506    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm" podUID="b38ebfe9-843d-4965-8da5-7fc858a0b4eb" containerName="sequence-graph-8996c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:17:07.378669 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:07.378630    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm" podUID="b38ebfe9-843d-4965-8da5-7fc858a0b4eb" containerName="sequence-graph-8996c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:17:09.219297 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.219267    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"]
Jul 07 15:17:09.222538 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.222519    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:09.225571 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.225540    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-542f9-kube-rbac-proxy-sar-config\""
Jul 07 15:17:09.225753 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.225735    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-542f9-serving-cert\""
Jul 07 15:17:09.234140 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.234117    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"]
Jul 07 15:17:09.234860 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.234841    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:09.280964 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.280932    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-openshift-service-ca-bundle\") pod \"ensemble-graph-542f9-767cbb57f5-vn4jq\" (UID: \"85fa4d2c-a3ca-4413-8fbe-252a0777e49f\") " pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:09.281127 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.280988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-proxy-tls\") pod \"ensemble-graph-542f9-767cbb57f5-vn4jq\" (UID: \"85fa4d2c-a3ca-4413-8fbe-252a0777e49f\") " pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:09.381463 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.381422    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-openshift-service-ca-bundle\") pod \"ensemble-graph-542f9-767cbb57f5-vn4jq\" (UID: \"85fa4d2c-a3ca-4413-8fbe-252a0777e49f\") " pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:09.381645 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.381506    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-proxy-tls\") pod \"ensemble-graph-542f9-767cbb57f5-vn4jq\" (UID: \"85fa4d2c-a3ca-4413-8fbe-252a0777e49f\") " pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:09.381645 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:17:09.381626    2569 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-542f9-serving-cert: secret "ensemble-graph-542f9-serving-cert" not found
Jul 07 15:17:09.381717 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:17:09.381697    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-proxy-tls podName:85fa4d2c-a3ca-4413-8fbe-252a0777e49f nodeName:}" failed. No retries permitted until 2026-07-07 15:17:09.881674997 +0000 UTC m=+1225.894071522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-proxy-tls") pod "ensemble-graph-542f9-767cbb57f5-vn4jq" (UID: "85fa4d2c-a3ca-4413-8fbe-252a0777e49f") : secret "ensemble-graph-542f9-serving-cert" not found
Jul 07 15:17:09.382175 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.382155    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-openshift-service-ca-bundle\") pod \"ensemble-graph-542f9-767cbb57f5-vn4jq\" (UID: \"85fa4d2c-a3ca-4413-8fbe-252a0777e49f\") " pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:09.771095 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.771048    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:09.886034 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.885993    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-proxy-tls\") pod \"ensemble-graph-542f9-767cbb57f5-vn4jq\" (UID: \"85fa4d2c-a3ca-4413-8fbe-252a0777e49f\") " pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:09.888365 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:09.888335    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-proxy-tls\") pod \"ensemble-graph-542f9-767cbb57f5-vn4jq\" (UID: \"85fa4d2c-a3ca-4413-8fbe-252a0777e49f\") " pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:10.133234 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:10.133154    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:10.265761 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:10.265660    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"]
Jul 07 15:17:10.309003 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:10.308971    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq" event={"ID":"85fa4d2c-a3ca-4413-8fbe-252a0777e49f","Type":"ContainerStarted","Data":"5f1967064a7b5c50ad0b516bccbbae44056391bb8280e5eb2f90062d19e076eb"}
Jul 07 15:17:11.315219 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:11.315184    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq" event={"ID":"85fa4d2c-a3ca-4413-8fbe-252a0777e49f","Type":"ContainerStarted","Data":"594e9f06ebd33d63a65450dd16bb00cc1dcf54a646699631027c979f2863c103"}
Jul 07 15:17:11.315723 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:11.315295    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:11.332657 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:11.332600    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq" podStartSLOduration=2.332580792 podStartE2EDuration="2.332580792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:17:11.33212771 +0000 UTC m=+1227.344524256" watchObservedRunningTime="2026-07-07 15:17:11.332580792 +0000 UTC m=+1227.344977389"
Jul 07 15:17:12.378030 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:12.377989    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm" podUID="b38ebfe9-843d-4965-8da5-7fc858a0b4eb" containerName="sequence-graph-8996c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:17:14.849639 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:14.849615    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:17:14.930523 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:14.930486    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b38ebfe9-843d-4965-8da5-7fc858a0b4eb-proxy-tls\") pod \"b38ebfe9-843d-4965-8da5-7fc858a0b4eb\" (UID: \"b38ebfe9-843d-4965-8da5-7fc858a0b4eb\") "
Jul 07 15:17:14.930699 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:14.930546    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b38ebfe9-843d-4965-8da5-7fc858a0b4eb-openshift-service-ca-bundle\") pod \"b38ebfe9-843d-4965-8da5-7fc858a0b4eb\" (UID: \"b38ebfe9-843d-4965-8da5-7fc858a0b4eb\") "
Jul 07 15:17:14.930895 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:14.930873    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b38ebfe9-843d-4965-8da5-7fc858a0b4eb-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b38ebfe9-843d-4965-8da5-7fc858a0b4eb" (UID: "b38ebfe9-843d-4965-8da5-7fc858a0b4eb"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:17:14.932577 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:14.932548    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b38ebfe9-843d-4965-8da5-7fc858a0b4eb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b38ebfe9-843d-4965-8da5-7fc858a0b4eb" (UID: "b38ebfe9-843d-4965-8da5-7fc858a0b4eb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:17:15.031119 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:15.031025    2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b38ebfe9-843d-4965-8da5-7fc858a0b4eb-proxy-tls\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:17:15.031119 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:15.031068    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b38ebfe9-843d-4965-8da5-7fc858a0b4eb-openshift-service-ca-bundle\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:17:15.332760 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:15.332667    2569 generic.go:356] "Generic (PLEG): container finished" podID="b38ebfe9-843d-4965-8da5-7fc858a0b4eb" containerID="49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957" exitCode=0
Jul 07 15:17:15.332760 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:15.332718    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"
Jul 07 15:17:15.332966 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:15.332753    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm" event={"ID":"b38ebfe9-843d-4965-8da5-7fc858a0b4eb","Type":"ContainerDied","Data":"49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957"}
Jul 07 15:17:15.332966 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:15.332806    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm" event={"ID":"b38ebfe9-843d-4965-8da5-7fc858a0b4eb","Type":"ContainerDied","Data":"202ed4b63e831a592d18b383a698e1f9fa220bfcbb3ff8842a14e5efcdd231c8"}
Jul 07 15:17:15.332966 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:15.332830    2569 scope.go:117] "RemoveContainer" containerID="49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957"
Jul 07 15:17:15.341272 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:15.341237    2569 scope.go:117] "RemoveContainer" containerID="49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957"
Jul 07 15:17:15.341526 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:17:15.341509    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957\": container with ID starting with 49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957 not found: ID does not exist" containerID="49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957"
Jul 07 15:17:15.341589 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:15.341538    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957"} err="failed to get container status \"49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957\": rpc error: code = NotFound desc = could not find container \"49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957\": container with ID starting with 49cde1435b1f051f6524d9648adeef7fdf68eed2b00e243dee5fc1979c162957 not found: ID does not exist"
Jul 07 15:17:15.353888 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:15.353867    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"]
Jul 07 15:17:15.360423 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:15.360403    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8996c-6b8f8959c8-dl6dm"]
Jul 07 15:17:16.516216 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:16.516180    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b38ebfe9-843d-4965-8da5-7fc858a0b4eb" path="/var/lib/kubelet/pods/b38ebfe9-843d-4965-8da5-7fc858a0b4eb/volumes"
Jul 07 15:17:17.324705 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:17.324680    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:19.315070 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:19.315022    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"]
Jul 07 15:17:19.315569 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:19.315355    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq" podUID="85fa4d2c-a3ca-4413-8fbe-252a0777e49f" containerName="ensemble-graph-542f9" containerID="cri-o://594e9f06ebd33d63a65450dd16bb00cc1dcf54a646699631027c979f2863c103" gracePeriod=30
Jul 07 15:17:19.347938 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:19.347907    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:19.548498 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:19.548467    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:19.653267 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:19.653179    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:19.750036 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:19.750008    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:19.849294 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:19.849262    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:22.324099 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:22.324061    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq" podUID="85fa4d2c-a3ca-4413-8fbe-252a0777e49f" containerName="ensemble-graph-542f9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:17:27.324023 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:27.323988    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq" podUID="85fa4d2c-a3ca-4413-8fbe-252a0777e49f" containerName="ensemble-graph-542f9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:17:32.323560 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:32.323525    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq" podUID="85fa4d2c-a3ca-4413-8fbe-252a0777e49f" containerName="ensemble-graph-542f9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:17:32.323935 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:32.323637    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:37.323957 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:37.323875    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq" podUID="85fa4d2c-a3ca-4413-8fbe-252a0777e49f" containerName="ensemble-graph-542f9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:17:42.323868 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:42.323822    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq" podUID="85fa4d2c-a3ca-4413-8fbe-252a0777e49f" containerName="ensemble-graph-542f9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:17:44.924663 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:44.924625    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"]
Jul 07 15:17:44.925026 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:44.924949    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b38ebfe9-843d-4965-8da5-7fc858a0b4eb" containerName="sequence-graph-8996c"
Jul 07 15:17:44.925026 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:44.924960    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b38ebfe9-843d-4965-8da5-7fc858a0b4eb" containerName="sequence-graph-8996c"
Jul 07 15:17:44.925026 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:44.925004    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b38ebfe9-843d-4965-8da5-7fc858a0b4eb" containerName="sequence-graph-8996c"
Jul 07 15:17:44.928898 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:44.928879    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:17:44.932615 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:44.932590    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-e4605-serving-cert\""
Jul 07 15:17:44.932803 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:44.932783    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-e4605-kube-rbac-proxy-sar-config\""
Jul 07 15:17:44.938346 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:44.938317    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"]
Jul 07 15:17:44.938549 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:44.938527    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:44.961557 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:44.961524    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b8ff846-ce7a-4130-bb6b-de6e50651965-openshift-service-ca-bundle\") pod \"sequence-graph-e4605-6f67fbb78f-8gb4g\" (UID: \"3b8ff846-ce7a-4130-bb6b-de6e50651965\") " pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:17:44.961719 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:44.961573    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b8ff846-ce7a-4130-bb6b-de6e50651965-proxy-tls\") pod \"sequence-graph-e4605-6f67fbb78f-8gb4g\" (UID: \"3b8ff846-ce7a-4130-bb6b-de6e50651965\") " pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:17:45.060068 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:45.060038    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:45.062287 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:45.062232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b8ff846-ce7a-4130-bb6b-de6e50651965-openshift-service-ca-bundle\") pod \"sequence-graph-e4605-6f67fbb78f-8gb4g\" (UID: \"3b8ff846-ce7a-4130-bb6b-de6e50651965\") " pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:17:45.062357 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:45.062324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b8ff846-ce7a-4130-bb6b-de6e50651965-proxy-tls\") pod \"sequence-graph-e4605-6f67fbb78f-8gb4g\" (UID: \"3b8ff846-ce7a-4130-bb6b-de6e50651965\") " pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:17:45.062891 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:45.062864    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b8ff846-ce7a-4130-bb6b-de6e50651965-openshift-service-ca-bundle\") pod \"sequence-graph-e4605-6f67fbb78f-8gb4g\" (UID: \"3b8ff846-ce7a-4130-bb6b-de6e50651965\") " pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:17:45.064586 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:45.064560    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b8ff846-ce7a-4130-bb6b-de6e50651965-proxy-tls\") pod \"sequence-graph-e4605-6f67fbb78f-8gb4g\" (UID: \"3b8ff846-ce7a-4130-bb6b-de6e50651965\") " pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:17:45.239978 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:45.239903    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:17:45.370078 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:45.370009    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"]
Jul 07 15:17:45.372480 ip-10-0-130-158 kubenswrapper[2569]: W0707 15:17:45.372454    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b8ff846_ce7a_4130_bb6b_de6e50651965.slice/crio-e31475d01f2b3a465a7efcc9292daee8910221b9233717479ef196eb9e5be740 WatchSource:0}: Error finding container e31475d01f2b3a465a7efcc9292daee8910221b9233717479ef196eb9e5be740: Status 404 returned error can't find the container with id e31475d01f2b3a465a7efcc9292daee8910221b9233717479ef196eb9e5be740
Jul 07 15:17:45.446587 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:45.446548    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g" event={"ID":"3b8ff846-ce7a-4130-bb6b-de6e50651965","Type":"ContainerStarted","Data":"9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1"}
Jul 07 15:17:45.446750 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:45.446609    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g" event={"ID":"3b8ff846-ce7a-4130-bb6b-de6e50651965","Type":"ContainerStarted","Data":"e31475d01f2b3a465a7efcc9292daee8910221b9233717479ef196eb9e5be740"}
Jul 07 15:17:45.446750 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:45.446654    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:17:45.465736 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:45.465686    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g" podStartSLOduration=1.465669587 podStartE2EDuration="1.465669587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:17:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:17:45.464673046 +0000 UTC m=+1261.477069644" watchObservedRunningTime="2026-07-07 15:17:45.465669587 +0000 UTC m=+1261.478066133"
Jul 07 15:17:47.324477 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:47.324433    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq" podUID="85fa4d2c-a3ca-4413-8fbe-252a0777e49f" containerName="ensemble-graph-542f9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:17:49.462297 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:49.462245    2569 generic.go:356] "Generic (PLEG): container finished" podID="85fa4d2c-a3ca-4413-8fbe-252a0777e49f" containerID="594e9f06ebd33d63a65450dd16bb00cc1dcf54a646699631027c979f2863c103" exitCode=0
Jul 07 15:17:49.462297 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:49.462289    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq" event={"ID":"85fa4d2c-a3ca-4413-8fbe-252a0777e49f","Type":"ContainerDied","Data":"594e9f06ebd33d63a65450dd16bb00cc1dcf54a646699631027c979f2863c103"}
Jul 07 15:17:49.503190 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:49.503166    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:49.595213 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:49.595126    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-openshift-service-ca-bundle\") pod \"85fa4d2c-a3ca-4413-8fbe-252a0777e49f\" (UID: \"85fa4d2c-a3ca-4413-8fbe-252a0777e49f\") "
Jul 07 15:17:49.595213 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:49.595186    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-proxy-tls\") pod \"85fa4d2c-a3ca-4413-8fbe-252a0777e49f\" (UID: \"85fa4d2c-a3ca-4413-8fbe-252a0777e49f\") "
Jul 07 15:17:49.595544 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:49.595521    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "85fa4d2c-a3ca-4413-8fbe-252a0777e49f" (UID: "85fa4d2c-a3ca-4413-8fbe-252a0777e49f"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:17:49.597246 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:49.597219    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "85fa4d2c-a3ca-4413-8fbe-252a0777e49f" (UID: "85fa4d2c-a3ca-4413-8fbe-252a0777e49f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:17:49.696209 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:49.696168    2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-proxy-tls\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:17:49.696209 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:49.696197    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85fa4d2c-a3ca-4413-8fbe-252a0777e49f-openshift-service-ca-bundle\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:17:50.466737 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:50.466693    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq" event={"ID":"85fa4d2c-a3ca-4413-8fbe-252a0777e49f","Type":"ContainerDied","Data":"5f1967064a7b5c50ad0b516bccbbae44056391bb8280e5eb2f90062d19e076eb"}
Jul 07 15:17:50.466737 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:50.466719    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"
Jul 07 15:17:50.467291 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:50.466786    2569 scope.go:117] "RemoveContainer" containerID="594e9f06ebd33d63a65450dd16bb00cc1dcf54a646699631027c979f2863c103"
Jul 07 15:17:50.493742 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:50.493718    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"]
Jul 07 15:17:50.496529 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:50.496506    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-542f9-767cbb57f5-vn4jq"]
Jul 07 15:17:50.518014 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:50.517991    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85fa4d2c-a3ca-4413-8fbe-252a0777e49f" path="/var/lib/kubelet/pods/85fa4d2c-a3ca-4413-8fbe-252a0777e49f/volumes"
Jul 07 15:17:51.455801 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:51.455771    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:17:55.010765 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:55.010733    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"]
Jul 07 15:17:55.011180 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:55.010924    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g" podUID="3b8ff846-ce7a-4130-bb6b-de6e50651965" containerName="sequence-graph-e4605" containerID="cri-o://9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1" gracePeriod=30
Jul 07 15:17:55.033398 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:55.033370    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:55.145136 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:55.145100    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:55.242349 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:55.242320    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:55.340957 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:55.340876    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:55.446925 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:55.446894    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:55.547868 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:55.547836    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:17:56.454310 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:17:56.454248    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g" podUID="3b8ff846-ce7a-4130-bb6b-de6e50651965" containerName="sequence-graph-e4605" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:18:01.454138 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:01.454099    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g" podUID="3b8ff846-ce7a-4130-bb6b-de6e50651965" containerName="sequence-graph-e4605" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:18:06.454226 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:06.454180    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g" podUID="3b8ff846-ce7a-4130-bb6b-de6e50651965" containerName="sequence-graph-e4605" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:18:06.454614 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:06.454335    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:18:11.453929 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:11.453891    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g" podUID="3b8ff846-ce7a-4130-bb6b-de6e50651965" containerName="sequence-graph-e4605" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:18:16.454220 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:16.454185    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g" podUID="3b8ff846-ce7a-4130-bb6b-de6e50651965" containerName="sequence-graph-e4605" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:18:19.574958 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:19.574924    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:18:19.800992 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:19.800957    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:18:21.454189 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:21.454147    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g" podUID="3b8ff846-ce7a-4130-bb6b-de6e50651965" containerName="sequence-graph-e4605" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:18:25.189841 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.189821    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:18:25.288165 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.288084    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b8ff846-ce7a-4130-bb6b-de6e50651965-openshift-service-ca-bundle\") pod \"3b8ff846-ce7a-4130-bb6b-de6e50651965\" (UID: \"3b8ff846-ce7a-4130-bb6b-de6e50651965\") "
Jul 07 15:18:25.288597 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.288183    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b8ff846-ce7a-4130-bb6b-de6e50651965-proxy-tls\") pod \"3b8ff846-ce7a-4130-bb6b-de6e50651965\" (UID: \"3b8ff846-ce7a-4130-bb6b-de6e50651965\") "
Jul 07 15:18:25.288597 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.288474    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b8ff846-ce7a-4130-bb6b-de6e50651965-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "3b8ff846-ce7a-4130-bb6b-de6e50651965" (UID: "3b8ff846-ce7a-4130-bb6b-de6e50651965"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:18:25.290106 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.290087    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b8ff846-ce7a-4130-bb6b-de6e50651965-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3b8ff846-ce7a-4130-bb6b-de6e50651965" (UID: "3b8ff846-ce7a-4130-bb6b-de6e50651965"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:18:25.388883 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.388844    2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b8ff846-ce7a-4130-bb6b-de6e50651965-proxy-tls\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:18:25.388883 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.388875    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b8ff846-ce7a-4130-bb6b-de6e50651965-openshift-service-ca-bundle\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:18:25.597620 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.597536    2569 generic.go:356] "Generic (PLEG): container finished" podID="3b8ff846-ce7a-4130-bb6b-de6e50651965" containerID="9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1" exitCode=137
Jul 07 15:18:25.597620 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.597577    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"
Jul 07 15:18:25.597842 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.597623    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g" event={"ID":"3b8ff846-ce7a-4130-bb6b-de6e50651965","Type":"ContainerDied","Data":"9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1"}
Jul 07 15:18:25.597842 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.597676    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g" event={"ID":"3b8ff846-ce7a-4130-bb6b-de6e50651965","Type":"ContainerDied","Data":"e31475d01f2b3a465a7efcc9292daee8910221b9233717479ef196eb9e5be740"}
Jul 07 15:18:25.597842 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.597706    2569 scope.go:117] "RemoveContainer" containerID="9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1"
Jul 07 15:18:25.605952 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.605934    2569 scope.go:117] "RemoveContainer" containerID="9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1"
Jul 07 15:18:25.606209 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:18:25.606184    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1\": container with ID starting with 9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1 not found: ID does not exist" containerID="9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1"
Jul 07 15:18:25.606269 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.606211    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1"} err="failed to get container status \"9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1\": rpc error: code = NotFound desc = could not find container \"9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1\": container with ID starting with 9bf91888b996e942ad8ff40c9e076dcbe6aeae65f53902929e1da55af8696cc1 not found: ID does not exist"
Jul 07 15:18:25.622769 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.622743    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"]
Jul 07 15:18:25.626331 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:25.626309    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e4605-6f67fbb78f-8gb4g"]
Jul 07 15:18:26.516739 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:26.516705    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b8ff846-ce7a-4130-bb6b-de6e50651965" path="/var/lib/kubelet/pods/3b8ff846-ce7a-4130-bb6b-de6e50651965/volumes"
Jul 07 15:18:55.253763 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.253732    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"]
Jul 07 15:18:55.254203 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.254040    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85fa4d2c-a3ca-4413-8fbe-252a0777e49f" containerName="ensemble-graph-542f9"
Jul 07 15:18:55.254203 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.254060    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="85fa4d2c-a3ca-4413-8fbe-252a0777e49f" containerName="ensemble-graph-542f9"
Jul 07 15:18:55.254203 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.254071    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b8ff846-ce7a-4130-bb6b-de6e50651965" containerName="sequence-graph-e4605"
Jul 07 15:18:55.254203 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.254077    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b8ff846-ce7a-4130-bb6b-de6e50651965" containerName="sequence-graph-e4605"
Jul 07 15:18:55.254203 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.254123    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b8ff846-ce7a-4130-bb6b-de6e50651965" containerName="sequence-graph-e4605"
Jul 07 15:18:55.254203 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.254131    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="85fa4d2c-a3ca-4413-8fbe-252a0777e49f" containerName="ensemble-graph-542f9"
Jul 07 15:18:55.258170 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.258151    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:18:55.263823 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.263800    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 07 15:18:55.263958 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.263854    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-gzwhn\""
Jul 07 15:18:55.264985 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.264962    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-a9c06-kube-rbac-proxy-sar-config\""
Jul 07 15:18:55.265376 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.265357    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-a9c06-serving-cert\""
Jul 07 15:18:55.272491 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.272469    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"]
Jul 07 15:18:55.281993 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.281955    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:18:55.317147 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.317110    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e903711c-e517-4c31-ab7c-108c7340d474-openshift-service-ca-bundle\") pod \"sequence-graph-a9c06-867dcbbc69-72thf\" (UID: \"e903711c-e517-4c31-ab7c-108c7340d474\") " pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:18:55.317362 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.317159    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e903711c-e517-4c31-ab7c-108c7340d474-proxy-tls\") pod \"sequence-graph-a9c06-867dcbbc69-72thf\" (UID: \"e903711c-e517-4c31-ab7c-108c7340d474\") " pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:18:55.417587 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.417551    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e903711c-e517-4c31-ab7c-108c7340d474-openshift-service-ca-bundle\") pod \"sequence-graph-a9c06-867dcbbc69-72thf\" (UID: \"e903711c-e517-4c31-ab7c-108c7340d474\") " pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:18:55.417756 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.417598    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e903711c-e517-4c31-ab7c-108c7340d474-proxy-tls\") pod \"sequence-graph-a9c06-867dcbbc69-72thf\" (UID: \"e903711c-e517-4c31-ab7c-108c7340d474\") " pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:18:55.417756 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:18:55.417719    2569 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-a9c06-serving-cert: secret "sequence-graph-a9c06-serving-cert" not found
Jul 07 15:18:55.417841 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:18:55.417781    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e903711c-e517-4c31-ab7c-108c7340d474-proxy-tls podName:e903711c-e517-4c31-ab7c-108c7340d474 nodeName:}" failed. No retries permitted until 2026-07-07 15:18:55.917766413 +0000 UTC m=+1331.930162937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/e903711c-e517-4c31-ab7c-108c7340d474-proxy-tls") pod "sequence-graph-a9c06-867dcbbc69-72thf" (UID: "e903711c-e517-4c31-ab7c-108c7340d474") : secret "sequence-graph-a9c06-serving-cert" not found
Jul 07 15:18:55.418182 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.418161    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e903711c-e517-4c31-ab7c-108c7340d474-openshift-service-ca-bundle\") pod \"sequence-graph-a9c06-867dcbbc69-72thf\" (UID: \"e903711c-e517-4c31-ab7c-108c7340d474\") " pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:18:55.921755 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.921718    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e903711c-e517-4c31-ab7c-108c7340d474-proxy-tls\") pod \"sequence-graph-a9c06-867dcbbc69-72thf\" (UID: \"e903711c-e517-4c31-ab7c-108c7340d474\") " pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:18:55.924041 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:55.924020    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e903711c-e517-4c31-ab7c-108c7340d474-proxy-tls\") pod \"sequence-graph-a9c06-867dcbbc69-72thf\" (UID: \"e903711c-e517-4c31-ab7c-108c7340d474\") " pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:18:56.003176 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:56.003139    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:18:56.167875 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:56.167834    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:18:56.300477 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:56.300449    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"]
Jul 07 15:18:56.302947 ip-10-0-130-158 kubenswrapper[2569]: W0707 15:18:56.302919    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode903711c_e517_4c31_ab7c_108c7340d474.slice/crio-74644f6915ad99d2da3fd076a3db1ff2e1acfd1c023d77dc2931f2a5707c746b WatchSource:0}: Error finding container 74644f6915ad99d2da3fd076a3db1ff2e1acfd1c023d77dc2931f2a5707c746b: Status 404 returned error can't find the container with id 74644f6915ad99d2da3fd076a3db1ff2e1acfd1c023d77dc2931f2a5707c746b
Jul 07 15:18:56.715470 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:56.715436    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf" event={"ID":"e903711c-e517-4c31-ab7c-108c7340d474","Type":"ContainerStarted","Data":"cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd"}
Jul 07 15:18:56.715634 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:56.715486    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf" event={"ID":"e903711c-e517-4c31-ab7c-108c7340d474","Type":"ContainerStarted","Data":"74644f6915ad99d2da3fd076a3db1ff2e1acfd1c023d77dc2931f2a5707c746b"}
Jul 07 15:18:56.715634 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:56.715515    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:18:56.734188 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:18:56.734135    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf" podStartSLOduration=1.734120414 podStartE2EDuration="1.734120414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:18:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:18:56.732860977 +0000 UTC m=+1332.745257545" watchObservedRunningTime="2026-07-07 15:18:56.734120414 +0000 UTC m=+1332.746516959"
Jul 07 15:19:02.724958 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:19:02.724932    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:20:58.697247 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:20:58.697155    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:25:57.642371 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:25:57.642339    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:26:34.262440 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:26:34.262382    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:26:34.579655 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:26:34.579583    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:26:34.705306 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:26:34.705278    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:26:34.834272 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:26:34.834174    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:26:34.981026 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:26:34.980994    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:09.829585 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:09.829551    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"]
Jul 07 15:27:09.830101 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:09.829782    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf" podUID="e903711c-e517-4c31-ab7c-108c7340d474" containerName="sequence-graph-a9c06" containerID="cri-o://cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd" gracePeriod=30
Jul 07 15:27:09.873997 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:09.873965    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:10.111312 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:10.111230    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:10.276426 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:10.276391    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:10.441166 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:10.441131    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:11.194394 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:11.194362    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:12.724156 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:12.724116    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf" podUID="e903711c-e517-4c31-ab7c-108c7340d474" containerName="sequence-graph-a9c06" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:27:17.723631 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:17.723587    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf" podUID="e903711c-e517-4c31-ab7c-108c7340d474" containerName="sequence-graph-a9c06" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:27:22.723617 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:22.723582    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf" podUID="e903711c-e517-4c31-ab7c-108c7340d474" containerName="sequence-graph-a9c06" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:27:22.724155 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:22.723695    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:27:27.724123 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:27.724089    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf" podUID="e903711c-e517-4c31-ab7c-108c7340d474" containerName="sequence-graph-a9c06" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:27:32.723557 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:32.723505    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf" podUID="e903711c-e517-4c31-ab7c-108c7340d474" containerName="sequence-graph-a9c06" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:27:34.516013 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.515982    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"]
Jul 07 15:27:34.519162 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.519146    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:27:34.522349 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.522332    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-761e8-serving-cert\""
Jul 07 15:27:34.522410 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.522369    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-761e8-kube-rbac-proxy-sar-config\""
Jul 07 15:27:34.536006 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.535982    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"]
Jul 07 15:27:34.551483 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.551461    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/94940e5d-2041-4d76-a7d5-3ad4a9d10b42-proxy-tls\") pod \"splitter-graph-761e8-d6767b7cc-qgk52\" (UID: \"94940e5d-2041-4d76-a7d5-3ad4a9d10b42\") " pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:27:34.551604 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.551489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94940e5d-2041-4d76-a7d5-3ad4a9d10b42-openshift-service-ca-bundle\") pod \"splitter-graph-761e8-d6767b7cc-qgk52\" (UID: \"94940e5d-2041-4d76-a7d5-3ad4a9d10b42\") " pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:27:34.556529 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.556494    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:34.652447 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.652414    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/94940e5d-2041-4d76-a7d5-3ad4a9d10b42-proxy-tls\") pod \"splitter-graph-761e8-d6767b7cc-qgk52\" (UID: \"94940e5d-2041-4d76-a7d5-3ad4a9d10b42\") " pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:27:34.652447 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.652447    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94940e5d-2041-4d76-a7d5-3ad4a9d10b42-openshift-service-ca-bundle\") pod \"splitter-graph-761e8-d6767b7cc-qgk52\" (UID: \"94940e5d-2041-4d76-a7d5-3ad4a9d10b42\") " pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:27:34.653056 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.653036    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94940e5d-2041-4d76-a7d5-3ad4a9d10b42-openshift-service-ca-bundle\") pod \"splitter-graph-761e8-d6767b7cc-qgk52\" (UID: \"94940e5d-2041-4d76-a7d5-3ad4a9d10b42\") " pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:27:34.654779 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.654762    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/94940e5d-2041-4d76-a7d5-3ad4a9d10b42-proxy-tls\") pod \"splitter-graph-761e8-d6767b7cc-qgk52\" (UID: \"94940e5d-2041-4d76-a7d5-3ad4a9d10b42\") " pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:27:34.777876 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.777801    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:34.829541 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.829516    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:27:34.989729 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:34.989687    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"]
Jul 07 15:27:34.996422 ip-10-0-130-158 kubenswrapper[2569]: W0707 15:27:34.996387    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94940e5d_2041_4d76_a7d5_3ad4a9d10b42.slice/crio-cdb6ffc7995093a88ecc68a3166100b5f426a46e0d8dcdadf5c5dc856ce483a9 WatchSource:0}: Error finding container cdb6ffc7995093a88ecc68a3166100b5f426a46e0d8dcdadf5c5dc856ce483a9: Status 404 returned error can't find the container with id cdb6ffc7995093a88ecc68a3166100b5f426a46e0d8dcdadf5c5dc856ce483a9
Jul 07 15:27:35.653052 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:35.653012    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52" event={"ID":"94940e5d-2041-4d76-a7d5-3ad4a9d10b42","Type":"ContainerStarted","Data":"6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603"}
Jul 07 15:27:35.653469 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:35.653067    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52" event={"ID":"94940e5d-2041-4d76-a7d5-3ad4a9d10b42","Type":"ContainerStarted","Data":"cdb6ffc7995093a88ecc68a3166100b5f426a46e0d8dcdadf5c5dc856ce483a9"}
Jul 07 15:27:35.653469 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:35.653126    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:27:35.678772 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:35.678723    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52" podStartSLOduration=1.6787079280000001 podStartE2EDuration="1.678707928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:27:35.677946624 +0000 UTC m=+1851.690343171" watchObservedRunningTime="2026-07-07 15:27:35.678707928 +0000 UTC m=+1851.691104526"
Jul 07 15:27:37.723269 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:37.723225    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf" podUID="e903711c-e517-4c31-ab7c-108c7340d474" containerName="sequence-graph-a9c06" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:27:39.976547 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:39.976520    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:27:39.998073 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:39.998050    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e903711c-e517-4c31-ab7c-108c7340d474-openshift-service-ca-bundle\") pod \"e903711c-e517-4c31-ab7c-108c7340d474\" (UID: \"e903711c-e517-4c31-ab7c-108c7340d474\") "
Jul 07 15:27:39.998187 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:39.998096    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e903711c-e517-4c31-ab7c-108c7340d474-proxy-tls\") pod \"e903711c-e517-4c31-ab7c-108c7340d474\" (UID: \"e903711c-e517-4c31-ab7c-108c7340d474\") "
Jul 07 15:27:39.998380 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:39.998360    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e903711c-e517-4c31-ab7c-108c7340d474-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "e903711c-e517-4c31-ab7c-108c7340d474" (UID: "e903711c-e517-4c31-ab7c-108c7340d474"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:27:39.999950 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:39.999925    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e903711c-e517-4c31-ab7c-108c7340d474-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e903711c-e517-4c31-ab7c-108c7340d474" (UID: "e903711c-e517-4c31-ab7c-108c7340d474"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:27:40.099245 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:40.099163    2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e903711c-e517-4c31-ab7c-108c7340d474-proxy-tls\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:27:40.099245 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:40.099190    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e903711c-e517-4c31-ab7c-108c7340d474-openshift-service-ca-bundle\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:27:40.672578 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:40.672541    2569 generic.go:356] "Generic (PLEG): container finished" podID="e903711c-e517-4c31-ab7c-108c7340d474" containerID="cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd" exitCode=0
Jul 07 15:27:40.672578 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:40.672582    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"
Jul 07 15:27:40.672793 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:40.672587    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf" event={"ID":"e903711c-e517-4c31-ab7c-108c7340d474","Type":"ContainerDied","Data":"cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd"}
Jul 07 15:27:40.672793 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:40.672644    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf" event={"ID":"e903711c-e517-4c31-ab7c-108c7340d474","Type":"ContainerDied","Data":"74644f6915ad99d2da3fd076a3db1ff2e1acfd1c023d77dc2931f2a5707c746b"}
Jul 07 15:27:40.672793 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:40.672668    2569 scope.go:117] "RemoveContainer" containerID="cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd"
Jul 07 15:27:40.680807 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:40.680779    2569 scope.go:117] "RemoveContainer" containerID="cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd"
Jul 07 15:27:40.681074 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:27:40.681052    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd\": container with ID starting with cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd not found: ID does not exist" containerID="cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd"
Jul 07 15:27:40.681152 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:40.681084    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd"} err="failed to get container status \"cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd\": rpc error: code = NotFound desc = could not find container \"cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd\": container with ID starting with cc8271ece81d715fb80c12266a78263c5fa15f57231e983c78d003b0a74091fd not found: ID does not exist"
Jul 07 15:27:40.700917 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:40.700890    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"]
Jul 07 15:27:40.713321 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:40.713295    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a9c06-867dcbbc69-72thf"]
Jul 07 15:27:41.663110 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:41.663082    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:27:42.516437 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:42.516406    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e903711c-e517-4c31-ab7c-108c7340d474" path="/var/lib/kubelet/pods/e903711c-e517-4c31-ab7c-108c7340d474/volumes"
Jul 07 15:27:44.582465 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:44.582440    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"]
Jul 07 15:27:44.582754 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:44.582641    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52" podUID="94940e5d-2041-4d76-a7d5-3ad4a9d10b42" containerName="splitter-graph-761e8" containerID="cri-o://6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603" gracePeriod=30
Jul 07 15:27:44.613699 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:44.613668    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:44.776659 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:44.776619    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:44.943302 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:44.943270    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:45.079299 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:45.079265    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:45.344122 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:45.344041    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:45.474202 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:45.474172    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:27:46.660802 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:46.660759    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52" podUID="94940e5d-2041-4d76-a7d5-3ad4a9d10b42" containerName="splitter-graph-761e8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:27:51.660728 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:51.660688    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52" podUID="94940e5d-2041-4d76-a7d5-3ad4a9d10b42" containerName="splitter-graph-761e8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:27:56.660793 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:56.660757    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52" podUID="94940e5d-2041-4d76-a7d5-3ad4a9d10b42" containerName="splitter-graph-761e8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:27:56.661278 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:27:56.660849    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:28:01.660318 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:01.660283    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52" podUID="94940e5d-2041-4d76-a7d5-3ad4a9d10b42" containerName="splitter-graph-761e8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:28:06.660581 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:06.660491    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52" podUID="94940e5d-2041-4d76-a7d5-3ad4a9d10b42" containerName="splitter-graph-761e8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:28:10.194225 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:10.194194    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:28:10.413048 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:10.413014    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:28:11.660474 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:11.660429    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52" podUID="94940e5d-2041-4d76-a7d5-3ad4a9d10b42" containerName="splitter-graph-761e8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:28:14.776950 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.776924    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:28:14.808002 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.807974    2569 generic.go:356] "Generic (PLEG): container finished" podID="94940e5d-2041-4d76-a7d5-3ad4a9d10b42" containerID="6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603" exitCode=0
Jul 07 15:28:14.808168 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.808028    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"
Jul 07 15:28:14.808168 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.808043    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52" event={"ID":"94940e5d-2041-4d76-a7d5-3ad4a9d10b42","Type":"ContainerDied","Data":"6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603"}
Jul 07 15:28:14.808168 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.808104    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52" event={"ID":"94940e5d-2041-4d76-a7d5-3ad4a9d10b42","Type":"ContainerDied","Data":"cdb6ffc7995093a88ecc68a3166100b5f426a46e0d8dcdadf5c5dc856ce483a9"}
Jul 07 15:28:14.808168 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.808134    2569 scope.go:117] "RemoveContainer" containerID="6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603"
Jul 07 15:28:14.816841 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.816814    2569 scope.go:117] "RemoveContainer" containerID="6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603"
Jul 07 15:28:14.817098 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:28:14.817079    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603\": container with ID starting with 6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603 not found: ID does not exist" containerID="6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603"
Jul 07 15:28:14.817155 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.817106    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603"} err="failed to get container status \"6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603\": rpc error: code = NotFound desc = could not find container \"6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603\": container with ID starting with 6e1e1cf156a27953f3eeb6fe1613096ed31ff89a7048defd49c514221a664603 not found: ID does not exist"
Jul 07 15:28:14.893547 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.893455    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/94940e5d-2041-4d76-a7d5-3ad4a9d10b42-proxy-tls\") pod \"94940e5d-2041-4d76-a7d5-3ad4a9d10b42\" (UID: \"94940e5d-2041-4d76-a7d5-3ad4a9d10b42\") "
Jul 07 15:28:14.893693 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.893583    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94940e5d-2041-4d76-a7d5-3ad4a9d10b42-openshift-service-ca-bundle\") pod \"94940e5d-2041-4d76-a7d5-3ad4a9d10b42\" (UID: \"94940e5d-2041-4d76-a7d5-3ad4a9d10b42\") "
Jul 07 15:28:14.893876 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.893855    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94940e5d-2041-4d76-a7d5-3ad4a9d10b42-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "94940e5d-2041-4d76-a7d5-3ad4a9d10b42" (UID: "94940e5d-2041-4d76-a7d5-3ad4a9d10b42"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:28:14.895515 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.895491    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94940e5d-2041-4d76-a7d5-3ad4a9d10b42-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "94940e5d-2041-4d76-a7d5-3ad4a9d10b42" (UID: "94940e5d-2041-4d76-a7d5-3ad4a9d10b42"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:28:14.994371 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.994336    2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/94940e5d-2041-4d76-a7d5-3ad4a9d10b42-proxy-tls\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:28:14.994371 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:14.994366    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94940e5d-2041-4d76-a7d5-3ad4a9d10b42-openshift-service-ca-bundle\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:28:15.140711 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:15.140680    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"]
Jul 07 15:28:15.150197 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:15.150130    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-761e8-d6767b7cc-qgk52"]
Jul 07 15:28:16.516696 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:16.516655    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94940e5d-2041-4d76-a7d5-3ad4a9d10b42" path="/var/lib/kubelet/pods/94940e5d-2041-4d76-a7d5-3ad4a9d10b42/volumes"
Jul 07 15:28:44.852664 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.852580    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"]
Jul 07 15:28:44.853126 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.852994    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94940e5d-2041-4d76-a7d5-3ad4a9d10b42" containerName="splitter-graph-761e8"
Jul 07 15:28:44.853126 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.853012    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="94940e5d-2041-4d76-a7d5-3ad4a9d10b42" containerName="splitter-graph-761e8"
Jul 07 15:28:44.853126 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.853038    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e903711c-e517-4c31-ab7c-108c7340d474" containerName="sequence-graph-a9c06"
Jul 07 15:28:44.853126 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.853046    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e903711c-e517-4c31-ab7c-108c7340d474" containerName="sequence-graph-a9c06"
Jul 07 15:28:44.853126 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.853112    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="94940e5d-2041-4d76-a7d5-3ad4a9d10b42" containerName="splitter-graph-761e8"
Jul 07 15:28:44.853126 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.853125    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e903711c-e517-4c31-ab7c-108c7340d474" containerName="sequence-graph-a9c06"
Jul 07 15:28:44.857196 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.857175    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:28:44.860006 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.859985    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-73e96-kube-rbac-proxy-sar-config\""
Jul 07 15:28:44.860104 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.860006    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-73e96-serving-cert\""
Jul 07 15:28:44.860163 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.860136    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-gzwhn\""
Jul 07 15:28:44.861144 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.861125    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 07 15:28:44.867976 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.867956    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"]
Jul 07 15:28:44.883140 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.883120    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:28:44.930815 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.930786    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ab77e72-be5f-4536-9acd-21e3d767e0f6-openshift-service-ca-bundle\") pod \"splitter-graph-73e96-7b66c9667c-dkq8v\" (UID: \"5ab77e72-be5f-4536-9acd-21e3d767e0f6\") " pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:28:44.930815 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:44.930819    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ab77e72-be5f-4536-9acd-21e3d767e0f6-proxy-tls\") pod \"splitter-graph-73e96-7b66c9667c-dkq8v\" (UID: \"5ab77e72-be5f-4536-9acd-21e3d767e0f6\") " pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:28:45.032147 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:45.032115    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ab77e72-be5f-4536-9acd-21e3d767e0f6-openshift-service-ca-bundle\") pod \"splitter-graph-73e96-7b66c9667c-dkq8v\" (UID: \"5ab77e72-be5f-4536-9acd-21e3d767e0f6\") " pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:28:45.032147 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:45.032149    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ab77e72-be5f-4536-9acd-21e3d767e0f6-proxy-tls\") pod \"splitter-graph-73e96-7b66c9667c-dkq8v\" (UID: \"5ab77e72-be5f-4536-9acd-21e3d767e0f6\") " pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:28:45.032353 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:28:45.032247    2569 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-73e96-serving-cert: secret "splitter-graph-73e96-serving-cert" not found
Jul 07 15:28:45.032353 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:28:45.032334    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ab77e72-be5f-4536-9acd-21e3d767e0f6-proxy-tls podName:5ab77e72-be5f-4536-9acd-21e3d767e0f6 nodeName:}" failed. No retries permitted until 2026-07-07 15:28:45.532316711 +0000 UTC m=+1921.544713238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/5ab77e72-be5f-4536-9acd-21e3d767e0f6-proxy-tls") pod "splitter-graph-73e96-7b66c9667c-dkq8v" (UID: "5ab77e72-be5f-4536-9acd-21e3d767e0f6") : secret "splitter-graph-73e96-serving-cert" not found
Jul 07 15:28:45.032775 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:45.032757    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ab77e72-be5f-4536-9acd-21e3d767e0f6-openshift-service-ca-bundle\") pod \"splitter-graph-73e96-7b66c9667c-dkq8v\" (UID: \"5ab77e72-be5f-4536-9acd-21e3d767e0f6\") " pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:28:45.311842 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:45.311808    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:28:45.536576 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:45.536538    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ab77e72-be5f-4536-9acd-21e3d767e0f6-proxy-tls\") pod \"splitter-graph-73e96-7b66c9667c-dkq8v\" (UID: \"5ab77e72-be5f-4536-9acd-21e3d767e0f6\") " pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:28:45.539056 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:45.539028    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ab77e72-be5f-4536-9acd-21e3d767e0f6-proxy-tls\") pod \"splitter-graph-73e96-7b66c9667c-dkq8v\" (UID: \"5ab77e72-be5f-4536-9acd-21e3d767e0f6\") " pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:28:45.767275 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:45.767215    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:28:45.896241 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:45.896122    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"]
Jul 07 15:28:45.898839 ip-10-0-130-158 kubenswrapper[2569]: W0707 15:28:45.898812    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ab77e72_be5f_4536_9acd_21e3d767e0f6.slice/crio-cb0b666113a21c798f567c593212d6efc94040735ef228520b8c6bc0408327ea WatchSource:0}: Error finding container cb0b666113a21c798f567c593212d6efc94040735ef228520b8c6bc0408327ea: Status 404 returned error can't find the container with id cb0b666113a21c798f567c593212d6efc94040735ef228520b8c6bc0408327ea
Jul 07 15:28:45.924219 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:45.924190    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v" event={"ID":"5ab77e72-be5f-4536-9acd-21e3d767e0f6","Type":"ContainerStarted","Data":"cb0b666113a21c798f567c593212d6efc94040735ef228520b8c6bc0408327ea"}
Jul 07 15:28:46.929692 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:46.929648    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v" event={"ID":"5ab77e72-be5f-4536-9acd-21e3d767e0f6","Type":"ContainerStarted","Data":"29a7947f9874c317626cf904a56ce624909084c8128fbab93b54c3ca1b17a2ed"}
Jul 07 15:28:46.930077 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:46.929784    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:28:46.960511 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:46.960461    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v" podStartSLOduration=2.960445259 podStartE2EDuration="2.960445259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:28:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:28:46.959828558 +0000 UTC m=+1922.972225105" watchObservedRunningTime="2026-07-07 15:28:46.960445259 +0000 UTC m=+1922.972841832"
Jul 07 15:28:52.939386 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:28:52.939358    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:29:07.211062 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:29:07.211026    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:30:13.696139 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:30:13.696104    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:32:29.264551 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:32:29.264509    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:32:52.068578 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:32:52.068499    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:32:52.174108 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:32:52.174075    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:35:00.425615 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:35:00.425578    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:36:59.523634 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:36:59.523598    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"]
Jul 07 15:36:59.524140 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:36:59.523784    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v" podUID="5ab77e72-be5f-4536-9acd-21e3d767e0f6" containerName="splitter-graph-73e96" containerID="cri-o://29a7947f9874c317626cf904a56ce624909084c8128fbab93b54c3ca1b17a2ed" gracePeriod=30
Jul 07 15:36:59.599182 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:36:59.599141    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:36:59.726561 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:36:59.726525    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:36:59.810907 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:36:59.810817    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:37:02.937983 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:02.937944    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v" podUID="5ab77e72-be5f-4536-9acd-21e3d767e0f6" containerName="splitter-graph-73e96" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:37:07.938818 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:07.938740    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v" podUID="5ab77e72-be5f-4536-9acd-21e3d767e0f6" containerName="splitter-graph-73e96" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:37:12.938851 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:12.938812    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v" podUID="5ab77e72-be5f-4536-9acd-21e3d767e0f6" containerName="splitter-graph-73e96" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:37:12.939219 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:12.938919    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:37:17.938444 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:17.938407    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v" podUID="5ab77e72-be5f-4536-9acd-21e3d767e0f6" containerName="splitter-graph-73e96" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:37:22.938272 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:22.938221    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v" podUID="5ab77e72-be5f-4536-9acd-21e3d767e0f6" containerName="splitter-graph-73e96" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:37:27.938777 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:27.938741    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v" podUID="5ab77e72-be5f-4536-9acd-21e3d767e0f6" containerName="splitter-graph-73e96" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:37:29.877097 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:29.877003    2569 generic.go:356] "Generic (PLEG): container finished" podID="5ab77e72-be5f-4536-9acd-21e3d767e0f6" containerID="29a7947f9874c317626cf904a56ce624909084c8128fbab93b54c3ca1b17a2ed" exitCode=0
Jul 07 15:37:29.877097 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:29.877041    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v" event={"ID":"5ab77e72-be5f-4536-9acd-21e3d767e0f6","Type":"ContainerDied","Data":"29a7947f9874c317626cf904a56ce624909084c8128fbab93b54c3ca1b17a2ed"}
Jul 07 15:37:30.173234 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:30.173211    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:37:30.297428 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:30.297386    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ab77e72-be5f-4536-9acd-21e3d767e0f6-openshift-service-ca-bundle\") pod \"5ab77e72-be5f-4536-9acd-21e3d767e0f6\" (UID: \"5ab77e72-be5f-4536-9acd-21e3d767e0f6\") "
Jul 07 15:37:30.297678 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:30.297457    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ab77e72-be5f-4536-9acd-21e3d767e0f6-proxy-tls\") pod \"5ab77e72-be5f-4536-9acd-21e3d767e0f6\" (UID: \"5ab77e72-be5f-4536-9acd-21e3d767e0f6\") "
Jul 07 15:37:30.297782 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:30.297758    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ab77e72-be5f-4536-9acd-21e3d767e0f6-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "5ab77e72-be5f-4536-9acd-21e3d767e0f6" (UID: "5ab77e72-be5f-4536-9acd-21e3d767e0f6"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:37:30.299470 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:30.299449    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ab77e72-be5f-4536-9acd-21e3d767e0f6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5ab77e72-be5f-4536-9acd-21e3d767e0f6" (UID: "5ab77e72-be5f-4536-9acd-21e3d767e0f6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:37:30.398727 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:30.398695    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ab77e72-be5f-4536-9acd-21e3d767e0f6-openshift-service-ca-bundle\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:37:30.398727 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:30.398722    2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ab77e72-be5f-4536-9acd-21e3d767e0f6-proxy-tls\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:37:30.881822 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:30.881737    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"
Jul 07 15:37:30.881822 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:30.881762    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v" event={"ID":"5ab77e72-be5f-4536-9acd-21e3d767e0f6","Type":"ContainerDied","Data":"cb0b666113a21c798f567c593212d6efc94040735ef228520b8c6bc0408327ea"}
Jul 07 15:37:30.882338 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:30.881829    2569 scope.go:117] "RemoveContainer" containerID="29a7947f9874c317626cf904a56ce624909084c8128fbab93b54c3ca1b17a2ed"
Jul 07 15:37:30.897558 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:30.897536    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"]
Jul 07 15:37:30.899676 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:30.899658    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-73e96-7b66c9667c-dkq8v"]
Jul 07 15:37:32.516338 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:37:32.516305    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ab77e72-be5f-4536-9acd-21e3d767e0f6" path="/var/lib/kubelet/pods/5ab77e72-be5f-4536-9acd-21e3d767e0f6/volumes"
Jul 07 15:41:25.561304 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:41:25.561247    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:41:50.136790 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:41:50.136706    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:44:29.446389 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:29.446352    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:44:29.579561 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:29.579521    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:44:29.661481 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:29.661446    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:44:30.211709 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.211677    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7jk5b/must-gather-qbz2w"]
Jul 07 15:44:30.211970 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.211959    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ab77e72-be5f-4536-9acd-21e3d767e0f6" containerName="splitter-graph-73e96"
Jul 07 15:44:30.212012 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.211972    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ab77e72-be5f-4536-9acd-21e3d767e0f6" containerName="splitter-graph-73e96"
Jul 07 15:44:30.212046 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.212020    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ab77e72-be5f-4536-9acd-21e3d767e0f6" containerName="splitter-graph-73e96"
Jul 07 15:44:30.214839 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.214822    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7jk5b/must-gather-qbz2w"
Jul 07 15:44:30.218067 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.218049    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7jk5b\"/\"default-dockercfg-4l9wj\""
Jul 07 15:44:30.218201 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.218092    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7jk5b\"/\"kube-root-ca.crt\""
Jul 07 15:44:30.219904 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.219889    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7jk5b\"/\"openshift-service-ca.crt\""
Jul 07 15:44:30.228890 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.228870    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7jk5b/must-gather-qbz2w"]
Jul 07 15:44:30.230490 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.230466    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:44:30.290857 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.290822    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m7dd\" (UniqueName: \"kubernetes.io/projected/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc-kube-api-access-2m7dd\") pod \"must-gather-qbz2w\" (UID: \"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc\") " pod="openshift-must-gather-7jk5b/must-gather-qbz2w"
Jul 07 15:44:30.290857 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.290865    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc-must-gather-output\") pod \"must-gather-qbz2w\" (UID: \"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc\") " pod="openshift-must-gather-7jk5b/must-gather-qbz2w"
Jul 07 15:44:30.323546 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.323514    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:44:30.391751 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.391719    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m7dd\" (UniqueName: \"kubernetes.io/projected/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc-kube-api-access-2m7dd\") pod \"must-gather-qbz2w\" (UID: \"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc\") " pod="openshift-must-gather-7jk5b/must-gather-qbz2w"
Jul 07 15:44:30.391919 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.391760    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc-must-gather-output\") pod \"must-gather-qbz2w\" (UID: \"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc\") " pod="openshift-must-gather-7jk5b/must-gather-qbz2w"
Jul 07 15:44:30.392067 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.392052    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc-must-gather-output\") pod \"must-gather-qbz2w\" (UID: \"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc\") " pod="openshift-must-gather-7jk5b/must-gather-qbz2w"
Jul 07 15:44:30.401550 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.401522    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m7dd\" (UniqueName: \"kubernetes.io/projected/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc-kube-api-access-2m7dd\") pod \"must-gather-qbz2w\" (UID: \"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc\") " pod="openshift-must-gather-7jk5b/must-gather-qbz2w"
Jul 07 15:44:30.523989 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.523919    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7jk5b/must-gather-qbz2w"
Jul 07 15:44:30.652650 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.652622    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7jk5b/must-gather-qbz2w"]
Jul 07 15:44:30.655166 ip-10-0-130-158 kubenswrapper[2569]: W0707 15:44:30.655140    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77c43c51_e2fa_4b95_a5c7_153e4d4d78bc.slice/crio-ba544cfba86cc66e677e36cf89ec840d21303a7385b8ca9dd83317bffa634838 WatchSource:0}: Error finding container ba544cfba86cc66e677e36cf89ec840d21303a7385b8ca9dd83317bffa634838: Status 404 returned error can't find the container with id ba544cfba86cc66e677e36cf89ec840d21303a7385b8ca9dd83317bffa634838
Jul 07 15:44:30.656830 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:30.656816    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:44:31.424888 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:31.424819    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7jk5b/must-gather-qbz2w" event={"ID":"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc","Type":"ContainerStarted","Data":"ba544cfba86cc66e677e36cf89ec840d21303a7385b8ca9dd83317bffa634838"}
Jul 07 15:44:37.453875 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:37.453839    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7jk5b/must-gather-qbz2w" event={"ID":"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc","Type":"ContainerStarted","Data":"68e5c833d78781cd930b365a16b507c526c5247ab9244f88627f1b3dcd0c35f2"}
Jul 07 15:44:37.453875 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:37.453887    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7jk5b/must-gather-qbz2w" event={"ID":"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc","Type":"ContainerStarted","Data":"cce223afd47b57c643a7da5e85bb31b52e4cee209c3a177effd0d9e398e7900e"}
Jul 07 15:44:37.471374 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:44:37.471327    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7jk5b/must-gather-qbz2w" podStartSLOduration=1.281728516 podStartE2EDuration="7.471312004s" totalImagesPullingTime="6.189583488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:44:37.470088881 +0000 UTC m=+2873.482485428" watchObservedRunningTime="2026-07-07 15:44:37.471312004 +0000 UTC m=+2873.483708549"
Jul 07 15:45:00.554227 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:00.554195    2569 generic.go:356] "Generic (PLEG): container finished" podID="77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" containerID="cce223afd47b57c643a7da5e85bb31b52e4cee209c3a177effd0d9e398e7900e" exitCode=0
Jul 07 15:45:00.554642 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:00.554273    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7jk5b/must-gather-qbz2w" event={"ID":"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc","Type":"ContainerDied","Data":"cce223afd47b57c643a7da5e85bb31b52e4cee209c3a177effd0d9e398e7900e"}
Jul 07 15:45:00.554642 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:00.554564    2569 scope.go:117] "RemoveContainer" containerID="cce223afd47b57c643a7da5e85bb31b52e4cee209c3a177effd0d9e398e7900e"
Jul 07 15:45:00.960173 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:00.960140    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-7jk5b_must-gather-qbz2w_77c43c51-e2fa-4b95-a5c7-153e4d4d78bc/gather/0.log"
Jul 07 15:45:01.435520 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:01.435490    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:45:01.619495 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:01.619461    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:45:01.697084 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:01.697008    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:45:04.717338 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:04.717299    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4rqtz_5f99558a-08a5-4532-a131-eb30f2278bb5/global-pull-secret-syncer/0.log"
Jul 07 15:45:04.862208 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:04.862174    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log"
Jul 07 15:45:05.070882 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:05.070798    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-158.ec2.internal_831e9ec52a3238957c7f1fb3d14b894f/haproxy/0.log"
Jul 07 15:45:06.411820 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.411788    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-7jk5b/must-gather-qbz2w"]
Jul 07 15:45:06.412209 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.411961    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-7jk5b/must-gather-qbz2w" podUID="77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" containerName="copy" containerID="cri-o://68e5c833d78781cd930b365a16b507c526c5247ab9244f88627f1b3dcd0c35f2" gracePeriod=2
Jul 07 15:45:06.414971 ip-10-0-130-158 kubenswrapper[2569]: E0707 15:45:06.414937    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-qbz2w\" is forbidden: User \"system:node:ip-10-0-130-158.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-7jk5b\": no relationship found between node 'ip-10-0-130-158.ec2.internal' and this object" podUID="77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" pod="openshift-must-gather-7jk5b/must-gather-qbz2w"
Jul 07 15:45:06.416460 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.416438    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-7jk5b/must-gather-qbz2w"]
Jul 07 15:45:06.579731 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.579706    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-7jk5b_must-gather-qbz2w_77c43c51-e2fa-4b95-a5c7-153e4d4d78bc/copy/0.log"
Jul 07 15:45:06.580078 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.580056    2569 generic.go:356] "Generic (PLEG): container finished" podID="77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" containerID="68e5c833d78781cd930b365a16b507c526c5247ab9244f88627f1b3dcd0c35f2" exitCode=143
Jul 07 15:45:06.649852 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.649828    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-7jk5b_must-gather-qbz2w_77c43c51-e2fa-4b95-a5c7-153e4d4d78bc/copy/0.log"
Jul 07 15:45:06.651420 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.651402    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7jk5b/must-gather-qbz2w"
Jul 07 15:45:06.802630 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.802593    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc-must-gather-output\") pod \"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc\" (UID: \"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc\") "
Jul 07 15:45:06.802787 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.802647    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2m7dd\" (UniqueName: \"kubernetes.io/projected/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc-kube-api-access-2m7dd\") pod \"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc\" (UID: \"77c43c51-e2fa-4b95-a5c7-153e4d4d78bc\") "
Jul 07 15:45:06.804187 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.804152    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" (UID: "77c43c51-e2fa-4b95-a5c7-153e4d4d78bc"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:45:06.804836 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.804812    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc-kube-api-access-2m7dd" (OuterVolumeSpecName: "kube-api-access-2m7dd") pod "77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" (UID: "77c43c51-e2fa-4b95-a5c7-153e4d4d78bc"). InnerVolumeSpecName "kube-api-access-2m7dd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:45:06.903747 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.903709    2569 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc-must-gather-output\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:45:06.903747 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:06.903738    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2m7dd\" (UniqueName: \"kubernetes.io/projected/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc-kube-api-access-2m7dd\") on node \"ip-10-0-130-158.ec2.internal\" DevicePath \"\""
Jul 07 15:45:07.167632 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:07.167601    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:45:07.351179 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:07.351141    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zk5l_feab3f31-3c5f-4cc5-abb6-cc0405cb862f/konnectivity-agent/0.log" limit=1024
Jul 07 15:45:07.585111 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:07.585039    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-7jk5b_must-gather-qbz2w_77c43c51-e2fa-4b95-a5c7-153e4d4d78bc/copy/0.log"
Jul 07 15:45:07.585507 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:07.585420    2569 scope.go:117] "RemoveContainer" containerID="68e5c833d78781cd930b365a16b507c526c5247ab9244f88627f1b3dcd0c35f2"
Jul 07 15:45:07.585507 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:07.585421    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7jk5b/must-gather-qbz2w"
Jul 07 15:45:07.593485 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:07.593428    2569 scope.go:117] "RemoveContainer" containerID="cce223afd47b57c643a7da5e85bb31b52e4cee209c3a177effd0d9e398e7900e"
Jul 07 15:45:08.516623 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:08.516590    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" path="/var/lib/kubelet/pods/77c43c51-e2fa-4b95-a5c7-153e4d4d78bc/volumes"
Jul 07 15:45:09.238292 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.238265    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-fmh8f_b5883f9f-14d7-4c88-a3da-9919293e9e69/kube-state-metrics/0.log"
Jul 07 15:45:09.274176 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.274145    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-fmh8f_b5883f9f-14d7-4c88-a3da-9919293e9e69/kube-rbac-proxy-main/0.log"
Jul 07 15:45:09.304784 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.304751    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-fmh8f_b5883f9f-14d7-4c88-a3da-9919293e9e69/kube-rbac-proxy-self/0.log"
Jul 07 15:45:09.601142 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.601060    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b65zf_b779b44e-0ea3-4708-bb55-0d8fe65e53d2/node-exporter/0.log"
Jul 07 15:45:09.623382 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.623352    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b65zf_b779b44e-0ea3-4708-bb55-0d8fe65e53d2/kube-rbac-proxy/0.log"
Jul 07 15:45:09.645151 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.645123    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b65zf_b779b44e-0ea3-4708-bb55-0d8fe65e53d2/init-textfile/0.log"
Jul 07 15:45:09.673054 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.673022    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-4khdx_0340b77d-0352-4cfd-966e-61c7c94860fe/kube-rbac-proxy-main/0.log"
Jul 07 15:45:09.698150 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.698125    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-4khdx_0340b77d-0352-4cfd-966e-61c7c94860fe/kube-rbac-proxy-self/0.log"
Jul 07 15:45:09.720469 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.720440    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-4khdx_0340b77d-0352-4cfd-966e-61c7c94860fe/openshift-state-metrics/0.log"
Jul 07 15:45:09.771403 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.771373    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_49785725-58dd-4f67-85d6-9a3a94080dda/prometheus/0.log"
Jul 07 15:45:09.790620 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.790595    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_49785725-58dd-4f67-85d6-9a3a94080dda/config-reloader/0.log"
Jul 07 15:45:09.814019 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.813976    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_49785725-58dd-4f67-85d6-9a3a94080dda/thanos-sidecar/0.log"
Jul 07 15:45:09.841093 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.841068    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_49785725-58dd-4f67-85d6-9a3a94080dda/kube-rbac-proxy-web/0.log"
Jul 07 15:45:09.865366 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.865303    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_49785725-58dd-4f67-85d6-9a3a94080dda/kube-rbac-proxy/0.log"
Jul 07 15:45:09.886936 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.886911    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_49785725-58dd-4f67-85d6-9a3a94080dda/kube-rbac-proxy-thanos/0.log"
Jul 07 15:45:09.914350 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:09.914313    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_49785725-58dd-4f67-85d6-9a3a94080dda/init-config-reloader/0.log"
Jul 07 15:45:12.940588 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:12.940553    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"]
Jul 07 15:45:12.940968 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:12.940831    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" containerName="copy"
Jul 07 15:45:12.940968 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:12.940841    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" containerName="copy"
Jul 07 15:45:12.940968 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:12.940868    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" containerName="gather"
Jul 07 15:45:12.940968 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:12.940874    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" containerName="gather"
Jul 07 15:45:12.940968 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:12.940948    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" containerName="gather"
Jul 07 15:45:12.940968 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:12.940958    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="77c43c51-e2fa-4b95-a5c7-153e4d4d78bc" containerName="copy"
Jul 07 15:45:12.944766 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:12.944743    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:12.948712 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:12.948693    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4xx6g\"/\"kube-root-ca.crt\""
Jul 07 15:45:12.948826 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:12.948729    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4xx6g\"/\"openshift-service-ca.crt\""
Jul 07 15:45:12.948826 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:12.948782    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4xx6g\"/\"default-dockercfg-s4f8t\""
Jul 07 15:45:12.952404 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:12.952384    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"]
Jul 07 15:45:13.046934 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.046905    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-podres\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.047099 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.046943    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hjhl\" (UniqueName: \"kubernetes.io/projected/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-kube-api-access-7hjhl\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.047099 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.046992    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-lib-modules\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.047099 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.047026    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-proc\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.047099 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.047076    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-sys\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.148343 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.148301    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-lib-modules\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.148522 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.148393    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-proc\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.148522 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.148421    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-sys\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.148522 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.148450    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-podres\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.148522 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.148477    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-proc\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.148522 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.148480    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hjhl\" (UniqueName: \"kubernetes.io/projected/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-kube-api-access-7hjhl\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.148682 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.148521    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-sys\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.148682 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.148477    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-lib-modules\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.148682 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.148578    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-podres\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.156965 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.156944    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hjhl\" (UniqueName: \"kubernetes.io/projected/a7d25c00-3ff6-491a-8f29-3ff6d95e5abf-kube-api-access-7hjhl\") pod \"perf-node-gather-daemonset-clfsc\" (UID: \"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf\") " pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.254746 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.254663    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.384307 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.384284    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"]
Jul 07 15:45:13.386806 ip-10-0-130-158 kubenswrapper[2569]: W0707 15:45:13.386775    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda7d25c00_3ff6_491a_8f29_3ff6d95e5abf.slice/crio-56d14c550f68d0c3e8891b13cd7fd66ca485a39a3550514c77be5c96ee1901f1 WatchSource:0}: Error finding container 56d14c550f68d0c3e8891b13cd7fd66ca485a39a3550514c77be5c96ee1901f1: Status 404 returned error can't find the container with id 56d14c550f68d0c3e8891b13cd7fd66ca485a39a3550514c77be5c96ee1901f1
Jul 07 15:45:13.612182 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.612079    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc" event={"ID":"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf","Type":"ContainerStarted","Data":"44b641199d4f34414d46ac6db9555f3ab5d1a7ae579c6d0a5bd092b46ff74339"}
Jul 07 15:45:13.612182 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.612145    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc" event={"ID":"a7d25c00-3ff6-491a-8f29-3ff6d95e5abf","Type":"ContainerStarted","Data":"56d14c550f68d0c3e8891b13cd7fd66ca485a39a3550514c77be5c96ee1901f1"}
Jul 07 15:45:13.612182 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.612175    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:13.629182 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:13.629132    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc" podStartSLOduration=1.629115603 podStartE2EDuration="1.629115603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:45:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:13.627623727 +0000 UTC m=+2909.640020275" watchObservedRunningTime="2026-07-07 15:45:13.629115603 +0000 UTC m=+2909.641512202"
Jul 07 15:45:14.038184 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:14.038145    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wxktg_aafa76ab-df1f-4aba-9736-16236535b661/dns/0.log"
Jul 07 15:45:14.065655 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:14.065627    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wxktg_aafa76ab-df1f-4aba-9736-16236535b661/kube-rbac-proxy/0.log"
Jul 07 15:45:14.171107 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:14.171080    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cmwns_03e07f30-8257-4a4b-bb01-042bb672a64c/dns-node-resolver/0.log"
Jul 07 15:45:14.744387 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:14.744353    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ftfzh_6aaa686c-76ef-4753-86e1-5df48442b8ce/node-ca/0.log"
Jul 07 15:45:16.068043 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:16.068013    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mvtdl_e5c5d52e-2d2c-429a-97e9-752108e3d730/serve-healthcheck-canary/0.log"
Jul 07 15:45:16.622136 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:16.622097    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zwztf_7d5b9610-2c56-40d7-a623-472a3224d65e/kube-rbac-proxy/0.log"
Jul 07 15:45:16.646087 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:16.646055    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zwztf_7d5b9610-2c56-40d7-a623-472a3224d65e/exporter/0.log"
Jul 07 15:45:16.669994 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:16.669968    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zwztf_7d5b9610-2c56-40d7-a623-472a3224d65e/extractor/0.log"
Jul 07 15:45:19.196702 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:19.196675    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-2d6jl_fe39be5d-395c-4103-988f-28ce910fd01f/s3-init/0.log"
Jul 07 15:45:19.214523 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:19.214495    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-2d6jl_fe39be5d-395c-4103-988f-28ce910fd01f/download-hf-model/0.log"
Jul 07 15:45:19.625336 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:19.625270    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4xx6g/perf-node-gather-daemonset-clfsc"
Jul 07 15:45:23.349694 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:23.349660    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-f8tqp_8ae9950b-2803-432d-946f-6b22adb23e62/migrator/0.log"
Jul 07 15:45:23.372902 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:23.372873    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-f8tqp_8ae9950b-2803-432d-946f-6b22adb23e62/graceful-termination/0.log"
Jul 07 15:45:23.725324 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:23.725286    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-9dkqn_d15c4fa5-ac8a-4969-9ed6-bdf1fa681576/kube-storage-version-migrator-operator/1.log"
Jul 07 15:45:23.726921 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:23.726893    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-9dkqn_d15c4fa5-ac8a-4969-9ed6-bdf1fa681576/kube-storage-version-migrator-operator/0.log"
Jul 07 15:45:24.713064 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:24.713034    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2xxrr_15cba5b8-1945-4d35-9762-b160c3d27bde/kube-multus/0.log"
Jul 07 15:45:24.767376 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:24.767345    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2nvp9_b78593d3-7844-497b-96a9-17cd45cb49aa/kube-multus-additional-cni-plugins/0.log"
Jul 07 15:45:24.794153 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:24.794121    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2nvp9_b78593d3-7844-497b-96a9-17cd45cb49aa/egress-router-binary-copy/0.log"
Jul 07 15:45:24.818147 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:24.818119    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2nvp9_b78593d3-7844-497b-96a9-17cd45cb49aa/cni-plugins/0.log"
Jul 07 15:45:24.844721 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:24.844697    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2nvp9_b78593d3-7844-497b-96a9-17cd45cb49aa/bond-cni-plugin/0.log"
Jul 07 15:45:24.867315 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:24.867290    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2nvp9_b78593d3-7844-497b-96a9-17cd45cb49aa/routeoverride-cni/0.log"
Jul 07 15:45:24.895922 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:24.895893    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2nvp9_b78593d3-7844-497b-96a9-17cd45cb49aa/whereabouts-cni-bincopy/0.log"
Jul 07 15:45:24.918517 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:24.918493    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2nvp9_b78593d3-7844-497b-96a9-17cd45cb49aa/whereabouts-cni/0.log"
Jul 07 15:45:25.376825 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:25.376795    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s5zzk_7fcdb256-eea5-428e-9492-2cd608fd8a23/network-metrics-daemon/0.log"
Jul 07 15:45:25.397727 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:25.397697    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s5zzk_7fcdb256-eea5-428e-9492-2cd608fd8a23/kube-rbac-proxy/0.log"
Jul 07 15:45:26.447650 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:26.447622    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpxk2_b02523f0-255e-4b20-8947-e3d553aa71c2/ovn-controller/0.log"
Jul 07 15:45:26.463879 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:26.463856    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpxk2_b02523f0-255e-4b20-8947-e3d553aa71c2/ovn-acl-logging/0.log"
Jul 07 15:45:26.490693 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:26.490665    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpxk2_b02523f0-255e-4b20-8947-e3d553aa71c2/kube-rbac-proxy-node/0.log"
Jul 07 15:45:26.514890 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:26.514864    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpxk2_b02523f0-255e-4b20-8947-e3d553aa71c2/kube-rbac-proxy-ovn-metrics/0.log"
Jul 07 15:45:26.533725 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:26.533698    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpxk2_b02523f0-255e-4b20-8947-e3d553aa71c2/northd/0.log"
Jul 07 15:45:26.557268 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:26.557230    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpxk2_b02523f0-255e-4b20-8947-e3d553aa71c2/nbdb/0.log"
Jul 07 15:45:26.580367 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:26.580344    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpxk2_b02523f0-255e-4b20-8947-e3d553aa71c2/sbdb/0.log"
Jul 07 15:45:26.642214 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:26.642187    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpxk2_b02523f0-255e-4b20-8947-e3d553aa71c2/ovnkube-controller/0.log"
Jul 07 15:45:27.924612 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:27.924573    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-989f8b8dc-4jf9k_9bdcf86f-3d19-4170-b895-e30423ad9812/check-endpoints/0.log"
Jul 07 15:45:27.948110 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:27.948087    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-5w2jq_213be297-4d57-4440-9531-e4121fe87b19/network-check-target-container/0.log"
Jul 07 15:45:29.037941 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:29.037913    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-8dg7c_cf7be83c-ef1c-4e9c-9940-5c2cfe0fa093/iptables-alerter/0.log"
Jul 07 15:45:29.743118 ip-10-0-130-158 kubenswrapper[2569]: I0707 15:45:29.743090    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-5c8ng_3418602d-7767-4f00-b89f-8903c6faf891/tuned/0.log"